Critical Reasoning and Generative AI

One of the formative experiences in my mathematics education was a time I struggled to compute an integral in a calculus class. After sweating over it and scribbling for several minutes, I showed the instructor what I believed to be the correct answer. She said, “that’s a negative number.” Nonplussed (sorry!), I said, “so?” She then added, “you’re trying to compute a volume.”

That lesson stuck with me for more decades that I care to count. You don’t need to know calculus to understand that a geometric volume can’t be less than zero. The broader lesson is to always apply simple techniques from critical reasoning to problem solving — especially when you are working with methods or technologies that you don’t fully understand or trust.

If you take this lesson to heart, you won’t be as vulnerable to hallucinations, systematic bias, and other pitfalls of generative AI.

For example, if a generative AI system like ChatGPT responds to you with a provocative statement, check its work. It only takes a moment to search the web for corroboration from reputable sources. And do the search yourself, even if the generative AI cites sources that it might also be hallucinating! Verifying that an answer is correct tends to be a lot easier than coming up with that answer (cf. P vs. NP), so take a “trust, but verify” approach.

If you are using generative AI system for reasoning, such as a mathematical proof or causal chain of events, challenge the individual steps of the response. Better yet, ask the AI the opposite of your original request. If you find that a system contradicts itself, you will learn to be less trusting.

Here is a real example using ChatGPT.

I asked ChatGPT to prove that no prime number ends in 7. Even if you are not a math major, it shouldn’t take too long to figure out that 7, 17, and 37 are all prime numbers.

But here is how ChatGPT responded:

In case you don’t see the error in this “proof”, it’s the claim that 20k + 17 is “clearly divisible by 17” — which is clearly not the case for most values of k.

But even if you didn’t see the flaw, you could have simply asked ChatGPT for primes that end in 7. It happily answers the following:

At least ChatGPT had the decency to apologize for its earlier mistake!

I am sure the future will bring us more reliable generative AI systems. But no technology will ever be perfect, and no technology will ever take away our responsibility to engage in critical reasoning.

I hope that these simple techniques help you take advantage of generative AI while avoiding its pitfalls. Remember, AI is just a tool. You are responsible for making your own decisions — and for their consequences.