I dont know about what fantasy concepts always have about free... The rules are simply: Free speech is given to those, that give that rights as well to others. Bang your heads about oppinions as much as you want, but threaten another one for his oppinion and your out.
Just the same as it is with religion. The rule for getting the right of freely choosing your religion, is doing so to others as well. If you choose an religion, whose oppinion it is, that members of all others religions must be threated, your out.
The moment you allow people to threat others for their oppinion, people hardly can mention their oppinions anymore freely. Want to bother around about numbers of foreigners and impact on job? No prob. But threaten and make people on purpose aggressive against them, and your out.
This rules are no governmental tyranny, but exist out of an democratic majority, who wants those rules to exist? You are totally free to get yourself a democratic majority to change that rules. Everything else is tyranny.