I heard someone say that, when they're upset with someone, they tell them, or it turns into a resentment.
No.
Let's separate the practical from the spiritual.
Resentment flows from resisting reality and attempting to impose my demands on it. Whether or not I'm resentful has to do with the underlying demands. Sure, I can win a battle and feel vindicated, but if the demand is still there, I will simply get upset the next time. In any case, it's not right to relieve myself by being a jerk and attempting to induce guilt or shame in another person.
Does that mean that boundaries don't need to be set?
Absolutely not: boundaries are vital. Remember, our attitude is distinct from our action. We can be utterly accepting of the fact in our attitude and utterly non-accepting in our action. I accept that students are going to submit work late, and I don't get angry, but I still don't mark their work if submitted late. Ironclad boundary; total acceptance.
The question with boundaries is firstly whether there's any point and secondly how.
Most of the time, there's no point in even trying to set a boundary, because the person is unwilling or unable to comply. Repeatedly setting the boundary simply causes upset, consternation, and division.
There are situations where I am very small, the system is very big, and trying to set a boundary is simply being a jerk. Example: trying to get an airline to change its policy on something just for me.
There are situations where I call the shots, and, here, I am allowed to define how things are going to operate. Sometimes others have input and there is negotiation or there are requests. Sometimes, not.
Then there's everything else: with the 'everything else', which accounts for the bulk of my relationships with others, it's best to navigate deftly and by consensus, with nudges given by expressing wishes. With reasonable people, boundaries are rarely necessary. If they're regularly necessary, why am I with them?
If I must set a boundary, the best way is a polite request.