What many people fail to realize is that libertarians see rights and 'right and wrong' as two different things. We believe that individuals have the right to discriminate against others due to freedom of association, but that doesn't mean we believe discrimination is the right decision. Many libertarians believe that abortion is a right because forcing a woman to remain pregnant against her will is not consistent with the principle of self-ownership, but that doesn't mean that we endorse abortion as a preferred form of birth control. Most libertarians oppose the existence of status offenses (by which I mean actions which are considered illegal due only to the ages of those involved), but that doesn't mean we advocate 15-year-old kids getting drunk and having sex.
Rights encompass the range of actions in which an individual can engage without being guilty of the initiation of force against others. Right and wrong (i.e. morality) are standards of behavior that individuals impose on themselves and by which other individuals and communities may choose to regard and even to judge those with whom they do business or otherwise encounter. Just because libertarians argue for the maximization of individual rights does not mean we eschew personal morality.