The difference comes when the person, whether or not incited by hate speech, does more than just talk, but takes the hatred a bit further and commits a criminal act, such as an attack upon a person or a place simply because it is associated with some group the attacker does not tolerate. So hate speech and hate crime are not equivalent and as such hate speech is protected by the constitution. Can we ban hate speech from our campuses? This is a touchy subject and one which has even gone to the Supreme Court. Late in the 20th Century many schools sought to limit hate speech on the campuses. It was felt that this would prevent a rise in harassment. Speech codes...

In most cases, policies are considered to vague to be enforced. Opponents of the codes described them as elements of censorship in colleges and universities. The court arguments continue to this day.
Until such a time as the Supreme Court can finally rule on hate speech, it is likely we will have to endure it in our campuses. What we have to be sure of is that we do not react to it, and never ever act upon whatever anger or violence it tends to invoke in us, especially against the speaker. They may have the right to speak, but we definitely have the right not to listen.