The concept of "authority" can be defined as the right to control someone. While the word is sometimes used in other contexts (i.e. "the leading authority on UFO sightings"), the relevant definition is "the right to control" rather than as an inexact synonym for "expert." It is also improper to use the word to mean merely the ability to control something (you don't have "authority" over your watch simply because you have the ability to set it.) The notion of authority is about the right to control, and this quite simply cannot actually exist because no one ever has the right to control anyone other than themselves.
If you insist upon using the word authority, use it exclusively to describe the unique relationship that one has with their own body. You have authority over you (because you own yourself), but you can never have legitimate authority over anyone else. It may appear that one has authority, but it is truly just an appearance, for while one may act like they have the right to control someone else, it is impossible for anyone to actually possess such a right.
Let's talk for a moment about what authority isn't. It isn't the relationship that an employer has with an employee. An employer offers to trade consideration (typically money) in exchange for labor, but this is an ongoing cycle of voluntary trades which either party has the right to terminate at any point. An employer does not have a right to control his employee and the employee does not have an obligation to obey his employer. The employee chooses to do as his employer requests in exchange for consideration.
Authority is also not the relationship between a landlord and a tenant. A landlord allows the (typically conditional) use of his property in exchange for some type of consideration, often money but sometimes labor. Once again this is a voluntary trade. A landlord does not have a right to control his tenant and the tenant does not have an obligation to obey his landlord. Rather, the tenant chooses to live within the guidelines stipulated by the landlord up until such time as he chooses to vacate the landlord's property. If the desires of the landlord and the tenant cannot be reconciled, the voluntary trade may be suspended and the two parties are free to go their separate ways.
The term "legitimate authority" is actually an impossible concept much like a square circle is an impossible object. It simply cannot exist. No one ever has a right to control another person and no one ever has an obligation to obey another person. This applies to the relationship between parents and children, it applies to the "relationship" between individuals and the state, and it applies to the relationships between all other individuals.
When one attempts to exercise authority, they are engaging in aggression against whoever they attempt to control. The distinction which must be made is whether the action in question is actually an attempt to control or merely an impolite request. The relevant factor is whether the person giving the order claims the right to enforce obedience. If I say "pass the salt" or "close the door," chances are I'm not going to pull out a gun if you don't comply. What I'm really doing is asking you to do something, but whether you do it or not is entirely up to you. Such requests (even if given as commands) are thus not aggression because noncompliance remains an option.
Conversely, if a state enforcer demands that you "get on the ground!" even as he draws his firearm, what is happening is most definitely an act of aggression because the attempt to control you is backed by lethal force. Unfortunately, life is not always so black and white. Those who attempt to exercise control over others don't usually draw a gun in order to make it crystal clear that their presumed authority is indeed aggression.
How can you tell if someone's demands constitute aggression rather than just rudeness? Ask yourself what will happen if you don't obey. Will the person attempt to hurt you or take your stuff? If the answer is yes, the command is aggression. If the answer is no, they're just being rude. If you don't know, ask. Say, "What happens if I don't comply?" In some cases, they may not have an answer. Most petty tyrants don't really have an enforcement plan in place, they just rely on people acquiescing to their assumed authority.
You own yourself. You have the exclusive right to control your body, your actions, and your property. Others may try to persuade you to do or not do certain things, but they have no right to actually control you or your actions. Anyone who implicitly threatens you by attempting to control you (i.e. by attempting to exercise authority over you) is guilty of aggressing against your self-ownership. Such aggression warrants self-defense in order to protect yourself from their attempts to control you.
My advice to you is to start evaluating whose commands you're afraid to ignore and why. Is it just because you're a nice person who doesn't want to make waves or is it because saying no creates a very real danger that the self-styled "authority" will use violence to enforce their will? Anyone who falls in the latter category is a dangerous aggressor against whom you must be prepared to use self-defense. No one has the right to control you. Keep repeating it to yourself: "No one has the right to control me; no one has the right to control me; no one has the right to control me!"
Now get out there and start saying NO!