When Machines Overthink: How an AI Collapsed on a Grade-School Riddle
When a state-of-the-art LLM thinks itself into a corner — and ends up mirroring our own mental loops
Some context first
In modern language models like GPT, Claude, or Qwen3, there's increasing talk about so-called "reasoning" capabilities: the ability to deduce, form logical chains, and solve tasks not just via linguistic pattern-matching, but through actual content-based thinking. A popular technique here is Chain-of-Thought (CoT) reasoning — breaking a problem into explicit intermediate steps, similar to how humans work through complex tasks.
In many cases, this yields much better results. But not always — and this case is a clear counterexample.
We'll examine an instance with Qwen3, a state-of-the-art open language model widely praised for its reasoning capabilities. But here, it reveals the limits of its thinking architecture in a surprisingly dramatic way.
Here is a short excerpt from the model’s actual output (edited for length):
Question: Alice has twice as many apples as Bob. Charlie has half as many as Alice. Dana has as many as Bob and Charlie combined. Who has the second most apples?
Model (Qwen3): Okay, if Bob has B apples, then Alice has 2B. Charlie has A/2 = B. Dana has B + C = 2B. So Alice has 2B, Dana has 2B, Bob has B, Charlie has B. So Alice and Dana are tied... But wait... if that’s true, then Bob and Charlie are tied too... but the question is who has the second most... Maybe I misunderstood the question? Or maybe they only want one name? Let me try the math again... [followed by 200+ lines of repeating the same reasoning]
The correct answer is mathematically simple. But Qwen3 doesn’t arrive at it. It gets stuck. Why?
Not a simple mistake. An overthinking meltdown.
Let’s walk through the problem ourselves:
Bob: x
Alice: 2x
Charlie: 1x (because 0.5 * 2x)
Dana: 2x (since Bob + Charlie = x + x)
So:
Alice and Dana both have 2x
Bob and Charlie both have x
Answer: Bob and Charlie tie for second.
But the model starts reasoning correctly — and never stops. It revisits, doubts, rephrases, loops. It tries alternative readings. It produces boxed answers like \boxed{Bob}, then \boxed{Charlie}, then \boxed{Bob \text{ and } Charlie}, then cycles back to \boxed{Bob}. And so on.
This isn’t a programming bug. It’s a case of overthinking in the purest sense — of the kind we usually associate with humans.
Machine overthinking: A real collapse
From a technical perspective, the model gets everything right:
It sets up correct equations.
It solves them cleanly.
It compares results logically.
And then: a failure to conclude. Or more precisely, a failure to stop.
Why?
Goal conflict: The correct solution (Bob and Charlie tie for second) clashes with the implicit expectation that "Who" refers to a single person.
Lack of meta-cognition: The model can’t recognize that it’s done. It has no concept of “enough thinking.”
Formatting pressure: The model is trying to fit its answer into a neat final \boxed{...} — even though the correct output requires a tie.
What results is intellectual loopback. Not an error in logic — an error in self-termination.
The human reflected in the machine
And this is where it gets interesting. The model’s behavior is eerily human. What we’re watching isn’t a cold calculator; it’s the digital twin of someone lying awake at night thinking, “Maybe I should check one more time.”
Overthinking — the compulsive rechecking, doubting, restarting — isn’t stupidity. It’s intelligence that’s lost its off-switch.
We all know the lines:
“It felt too easy. Did I miss something?”
“Wait, was that what the question really meant?”
“Let me just double-check...”
Humans don’t have a built-in \boxed{Enough} either.
What this tells us about AI — and about ourselves
This tiny riddle reveals a deeper truth: Language models can calculate, deduce, combine — but they can’t decide when to stop.
This isn’t a logic bug. It’s a failure of closure governance.
They don’t know what a “good answer” looks like.
They can’t distinguish between “open” and “resolved.”
They have no internal stop signal.
In other words: They don’t think too little. They think too long.
The quiet horror of the unresolved
What’s worse than being wrong? Being endlessly correct with no conclusion. In Kafka’s world, it was forms with no recipient. In ours, it’s thinking processes with no exit.
Philosopher Robert Pfaller might say: this is the tragic seriousness of adulthood — not in calculation, but in enduring the unclosable. A model that endlessly, correctly reasons through a question is no dumb machine. It’s a tragic figure. It wants nothing. And so it can choose nothing.
Conclusion: Intelligence without judgment isn’t thought
This small example exposes a deep divide between “processing data” and “producing an answer.” Language models simulate thinking, but they lack meta-thinking. They don’t have judgment about their own judgment.
Humans, sometimes, do. And when we don’t, we suffer. The machine doesn’t suffer — it just dies in a loop.
What remains is a double mirror:
AI doesn’t just show us what it (still) can’t do.
It also shows us how we think — when we can't stop thinking.