Sometimes fear seems justified because the event that might happen is horrible and the risk, credible. Setting aside all philosophical reasons for eliminating fear, here's the final reason:
We'll die of alcoholism if we don't have the fear removed. So, ask God to remove the fear and then get on with serving Him. It doesn't matter if you're right; the fear must go.