Written language is (should be) carefully refined, looked over at least once after completion and tweaked.
Spoken language is (should be) extemporaneous and loose, or else the speaker sounds like a liar or a robot or both. That’s why things that are not acceptable in written language — like using “like” or “so” as stalling tactics while your brain catches up to what you want to say — are perfectly reasonable, even effective communication devices, in the context of spoken language. (The unspoken but meaningful subtext is often: “I said ‘like’ or ‘so’ because the next part of what I want to say is a little difficult to formulate, and communicating that to you is part of what I am communicating overall.”)
I have a lot of friends and acquaintances, especially editors and writers, who expect spoken language to be as well-tuned and elegantly tricked out as a New Yorker piece. In particular they decry stalling words like “like” and “so.” Or at least they claim to. I don’t know if the use of these words in these ways really bothers them, or if, like most “pet peeves,” their complaint is just a convenient gambit to have handy in case conversation lags — a sort of intellectual tchotchke designed to make its owner look more interesting. I suspect the latter. I sometimes run with a fairly pretentious, persnickety crowd — and I love them for it! I’m a bitch, too, just in other ways.
The English language is not settled science, subject to the rules of logic and consistency that pedants wish to impose upon it. It is a performance, subject only to the context in which it is presented and the needs and expectations of the speaker and the audience. The pedant’s “proper English” is appropriate in formal contexts, even when speaking. Let’s say you’re addressing the United Nations. You’d want to avoid “like” and “so” as stalling words. But I’m not going to use subjunctive verb phrases when talking to my mom, though, nor will “whom” (nor “nor”) ever cross my lips when I’m hanging outside with a bunch of silly queens at a gay bar, unless I am making fun of pretension itself. I’ll use other words and speak in other ways at other times.