Thinking critically (part 2)

So what exactly does it mean, to “think critically?”

It might be tempting to say that, since a lack of critical thinking more or less means accepting whatever you hear without question, then critical thinking would mean automatically rejecting everything you hear, again without question. But that’s not it; that’s just being difficult. (Side note: if you haven’t seen it, there’s a great Monty Python sketch called “The Argument Clinic” which is worth watching; you can see it on YouTube.)

Anyway, critical thinking means (to me at least), that you don’t automatically accept what you are told, but that you stop, take a moment, and evaluate what you hear. You compare it to what you know (or think you know), and then consciously decide to either accept it or not.

But that doesn’t quite get it either; if you’re comparing something to what you already know (or believe) to be true, there’s always the potential that you’re wrong. “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” (That great comment is usually credited to Mark Twain, but he probably was just quoting someone else!) So critical thinking means you also challenge your own assumptions and preconceived notions. This can be really challenging and forces you to be always on your guard against “what you know for sure that just ain’t so.” Always being on the lookout for your own blind spots can be really tiresome!

So how do you evaluate some new factoid, knowing that you may be walking into one of your own blind spots? Well, for some things I think it’s pretty straightforward. I work in a branch of healthcare science (nutrition) and I’m constantly being asked to evaluate claims made by competitors. Our customers (doctors) are surprisingly willing to accept outrageous claims on faith, which tells me that a lack of critical thinking isn’t limited to laypeople!

Anyhow, I teach our reps a three-step process to evaluate claims like this, and strongly recommend they teach the customer to do the same. It goes like this:

1) Show me the science.
2) If there’s no science, tell me about the historical use of the product.
3) If there’s not much history, then what’s the rationale for the product?

To elaborate: 1) What’s the science in support of the claim? What studies have been published? How were they conducted?? (Double-blind placebo-controlled crossover? Epidemiological? Case study? What?) Were they published? In what journals? How long ago? The claims can then be evaluated on the merits of the documentation. 2) If the science is not there or is weak, that doesn’t automatically cause the claims to be false, if there’s a history of application. For example, many herbs today have little direct science simply because western medicine hasn’t gotten around to studying them, but there’s a rich history of use in indigenous cultures. Chinese medicine, for example, records the use of specific herbs over a period spanning thousands of years. That’s a pretty large epidemiological study in anyone’s book! And 3) If there’s not a historical record of use and the science is spotty, then the explanation of what the product does had better match known scientific principles and it should “make sense” biochemically.

And with each failed step, my skeptic meter goes up a notch or two. Admittedly this is not fool proof (even scientists get bamboozled every now and then!) But this simple process has proven to be the most effective way I’ve come across to sift the wheat from the chaff of health claims.

That works pretty well for things that can be empirically studied, but what about “softer” claims, or purported facts from politics, sociology, etc. where actual studies don’t exist? Or where study design doesn’t lead to a definitive finding? How is critical thinking applied there?

Stay tuned for part 3!