Pause

Do I have to respond straight away?

No, unless there is a genuine urgency (e.g. a train will leave, a filing deadline will be missed, action has to be taken).

Does other people's anger or impatience constitute urgency?

No.

I'm always allowed to think before I respond, and if there is no urgency I will take the time I want.