Freedom of speech, as enshrined in the First Amendment of the United States Constitution, means that the government itself is not allowed to impair your ability to say whatever it is you want to say — within certain boundaries laid down by various Supreme Courts over the years. The government can prosecute you for shouting ‘fire’ in a crowded theater (um, maybe), for example, or for uttering ‘fighting words,’ whatever those are.
Freedom of speech does not mean that you can say whatever you want to say without consequence. It just protects you from legal consequence. Non-governmental consequences often follow controversial or inappropriate statements. For example, if you smart off to your cranky old grandmother, she might slap you in the face. If you submit a plagiarized story to your editor, you might get fired from your job as a reporter. If you make up stuff in your memoir, you might make Oprah angry. In none of these cases will you go to jail.
More to the point: if you are a prominent and raging anti-gay activist, I’m not going to pay money to see your movie, even if the movie has nothing to do with your anti-gay activism or your rage. The government isn’t involved in this decision of mine. The First Amendment doesn’t apply. I’m also going to be encouraging everybody I know to Skip Ender’s Game, which is my own exercise of my own free speech rights! Ta da! See how it works!
Speech has consequences, because speech matters. It would be a terrible world to live in if the things we say were completely irrelevant all the time, because they were “just words” or “just opinions” or “just” whatever. There’s no “just” about it. Words and opinions are powerful; they matter, and like anything that matters, they have consequences. There would be no reason to speak at all, otherwise. A world where anything can be said because nobody cares is a world where nothing is really said — and that’s the opposite of a world with free speech.
I don’t understand why this is a difficult concept for Orson Scott Card or his whiny-ass fans to understand.