A new research paper titled “AI Assistance Reduces Persistence and Hurts Independent Performance,” which was recently presented at the Conference on Language Modeling, has been attracting attention.
It explores an obvious question: do AI tools negatively affect your ability to think? It does so with a simple experimental design: participants were divided into two groups tasked with solving a series of math problems involving fractions. One group had to find answers on their own, while the other used ChatGPT to help.
At first, the ChatGPT group performed better, which is not surprising because LLMs are good at solving math problems. But then the researchers turned off the chatbot. Not only did the AI group’s accuracy fall below the control group, but many participants in the AI group gave up and started skipping questions altogether.
The researchers repeated the same experiment several more times, including one instance where math problems were replaced with an SAT reading comprehension prompt. They kept finding the same drop-offs in accuracy and persistence.
Here’s how a press release about the paper summarized these results:
“The findings backed up what anecdotes and surveys have increasingly found in recent years: An overreliance on AI for answers erodes our cognitive fitness. When AI produces answers in seconds, we get used to that speed; anything that takes longer feels inefficient.”
Brian Christian, one of the paper’s authors, put it plainly: “You almost couldn’t ask for a clearer story.”
I’ve spent the last decade thinking and writing about how technologies might accidentally degrade our cognitive fitness, and how this might in turn degrade not only our economic productivity, but also the quality of the human experience more generally.
In this sense, these LLM findings aren’t revealing something new.
What is new, however, is the jaw-dropping speed of these impacts. We’re accustomed to slow-moving techno-cognition trends such as the negative effects of ubiquitous smartphone usage, which took years to emerge in the data.
With chatbots, the negative impact is immediate. The high generated by side-stepping mental effort using LLMs is so compelling that, with only a short exposure, hard thinking quickly becomes intolerable.
The good news is that we don’t yet know if these effects are permanent. Presumably, the research participants who struggled immediately after ChatGPT was turned off might do much better on the same type of test if they tried again later in a setting where AI help was clearly not an option.
But we cannot ignore the warnings that these early studies are generating. When it comes to our cognitive fitness, we’re playing with a powerful new force that we don’t yet fully understand.
A key theme of my work in the technology critic space is that a life without serious, sustained, intentional thought is impoverished. Thinking is how we make sense of ourselves and the world, and is how we imagine new horizons and create the key elements of our future. It’s at the heart of philosophy, theology, and human empathy; the ability that separates our species from all other animals.
Silicon Valley is good at creating tools, but terrible at figuring out how we should use them. The latter step is still up to us — but it requires us to care.
Now do this same study with calculators. This is nothing new.
Dont think Calculators can be compares with AI…Its impact on current generations and for the generations to come will be profound
Agre here
For such a study you could read The Shallows by Nick Carr: calculators take cognitive load off your working memory freeing you up for more high-level thinking. Carr was talking about the internet more broadly, but it also applies to chatbots: they take the higher-level “deep work” executive tasks off you and dump more load onto your working memory, so it trains the wrong part of your brain.
Exactly. Using a calculator still requires one to know how to frame and enter the problem. As AI LLMs improve, increasingly sloppy prompts can eventually (but relatively quickly) lead to a correct answer. “Suggested follow up” prompts are eliminating the need for humans to ask good questions.
Well said, every bit of effort lost, and not replaced is less fitness.
James is right about the trend.
But…
TikTok is worse than TV tho. And LLMs TikTokify knowledge.
That being said, I find Lumo very useful with cognitive controls on it like timer blocks and a seperate machine so I can’t copy paste…
Graduated faster and with a 4.0 in part because of it.
Excellent article, Cal.
There is a subset of people who grew up being taught about the importance of hard work. There are those who dedicate their lives to the pursuit of knowledge just for the love of it. Buddhist followers are taught about the importance of struggles and obstacles to develop their strength and resilience on a mental and spiritual level.
I have not seen any evidence in all my travels that mankind will go extinct from LLM usage. In my opinion, I believe that people with illogical behaviours and thinking will continue to do so. The ones who pursue knowledge, thrive on cognitive struggles, and value hard work will continue to drive the world forward.
Although I have used LLMs to shortcut some of my tedious tasks, I have only ever looked at them like tools akin to an impact drill or pressure washer. They are advanced for sure, but they kill the pleasure of doing things with my own hands and critical thinking. I’m sure I’m not alone in this.
Is it concerning how it will impact everybody else? Certainly. But to be honest, I would let sleeping dogs lie.
These chatbots are not “pressure washers” and they’re certainly not being sold this way. A pressure washer is not something a human could never do. Thinking and communicating with others is fundamental to being human.
I think Cal is trying to help everyone and I wholeheartedly agree with the efforts. What happens to others (and society) matters for them and even us. Maybe “you can’t make a horse drink water” although how would we feel if members of our family and community were impacted? How will one feel if one is continuing to grow cognitively and one is working with many others who have gone the other direction? Will these people still be treated with dignity as they should be?
Cal,
Cognitive fitness might be increasingly rare due to reliance on LLMs, but is it still valuable in the workplace?
To me, a large part of why So Good They Can’t Ignore You is so great is because it highlights how cognitively intense careers are also usually well paid and remarkable to others. If that’s increasingly not the case, even if it will only last for a brief moment, then it deserves a mention.
Hey Henry,
His last paragraph answers your question.
“Silicon Valley is good at creating tools, but terrible at figuring out how we should use them. The latter step is still up to us — but it requires us to care.”
I would guess that it depends on your job, your sensitivity to stimulus, and ability to contain it’s creep.
What do you think? How would you control it to best serve you?
Cooked🔥
Any mindful knowledge worker can see this. Especially with programming.
LLMs are good for rapid fire encyclopedic knowledge, but even then, it’s a tiktokified dopamine siphon🚬. SHOUTOUT to Lumo 🔒🔏, encrypted and private, unlike all the others.
Frankly, all difficulty removed leads to a smaller AMCC (elastic willpower brain center).
Temp dopamine dip + less willpower = negative feedback loop.
This impacts every aspect of life, much like web surfing, bit magnified.
Stay vigilant. 🚭📵 📖📚
🔥🗞 Newport!
I think this is the part of AI that makes me most uneasy. I use it a lot — probably more than I should — because it is extraordinarily useful.
But that usefulness is also the problem. Friction disappears so quickly that you may not notice what the friction was doing for you.
I don’t think the answer is abstinence. For me the interesting question is which struggles I should deliberately keep. Reading something difficult. Writing the first draft myself. Sitting with a problem for a while before asking for help.
AI should remove drudgery. I’m less sure it should remove the parts that actually make us better at thinking.
So we move to designing our working process and how we interface with the tools we use ya?
What does that look like for you (open question for anyone here)?
For me:
– still spending time on paper, away from the computer
– writing first drafts, especially for communication with coworkers
– building kids into Claude to invite me to sit with the boredom while I wait for an answer or a build instead of starting a new session (managing context switching well)
– taking one day a week where I don’t use AI
– learning to turn work off (an even bigger challenge now)
For context: I’m a product designer that now is capable of building what I always had to depend on engineers to build for me. I work in a software start up (not in Silicon Valley).
I think before using AI for any math problem (or any hard problem for that matter), everyone should struggle by themselves for a couple of minutes.
That friction is important because you can pinpoint exactly where you struggle, and that place where you struggle is usually different from the place where another one struggles.
That cognitive struggle is important, because you can observe your thought process.
I remember talking with a student who explained to me she time-blocked her use of AI: every day at 4pm, because she wanted her brains to struggle a bit before she asked for help. I thought this was brilliant.
Great article Cal!
I’d really love a discussion about how to adequately use these LLMs. I’ve been essentially coerced by the fact that other peers use it so in order to keep the theater going at work I’m having to use it as well. Meanwhile, I’ve also realized that I’m more exhausted than before due to having to juggle so many threads and the context switching is killing me. The other main problem is that I’m progressively becoming more burned out, I stop caring about the outputs of these LLMs.
Unrelated, thank you for pointing me to More Everything Forever by Adam Becker. The book is fantastic, it’s helping me understand these pathological people in Silicon Valley.
I agree with most of what people are saying. However, I’d like to leave room for a skeptical take. What if friction isn’t actually disappearing but moving to a higher level of abstraction, at least for some individuals?
I don’t do arithmetic by hand, not because I want to reduce friction, but because I want to devote mental cycles to other, more interesting questions. I do get quick answers from LLMs, but that has forced me to think about how to formulate harder, more precise, and more meaningful questions.
Nothing new as one of the commenters above said. Calculators do the same thing.
Now we need the study that shows interacting with an Ai gives you new ideas and takes you in new directions that you might not have considered before. That happens all the time when using Ai, at least for me.
Calculating fractions and other trivial tasks are irrelevant.