It can be a little arbitrary, I admit. But usually if what you're about to say is pretty insulting and personal, it might be best to hold your tongue, especially if you're talking to someone in authority. Generally it's all about picking up on the right social cues.