Defining AI Psychosis. Part 2: "Prolific AI Psychosis"

Defining AI Psychosis. Part 2: "Prolific AI Psychosis" AI tools are one part senior engineer and one part toddler-running-across-white-carpet-with-a-jug-of-red-Kool-Aid. Your job is to determine which is which. Last week, I suggested that the term AI psychosis is applied to at least three different situations: 1) Genuine psychotic experiences related to LLM use, a phenomenon I described as “true AI psychosis.” 2) Hyperengagement with AI tools that is accompanied by a softer disconnection from reality. I’m calling this “prolific AI psychosis.” 3) Maladaptive relationships with AI chatbots that mimic human connection. I’m calling this “parasocial AI psychosis.” I then discussed true AI psychosis. I clarified what psychosis means to a psychiatrist. And I concluded that true AI psychosis is generally a variant of pre-existing psychosis rather than a new syndrome. This week, I’ll describe my current understanding of prolific AI psychosis. Prolific AI Psychosis Prolific AI psychosis occurs when a person generates a large quantity of AI output without significantly increasing the real value of their work. In some cases, the new AI workflow may even destroy value. For example, a software engineer with prolific AI psychosis can produce thousands of lines of code every day, but the code itself has little real-world utility. In contrast, a productive developer may create less code, but the created code is valuable to users and to their organization. Some days will include large additions to the codebase, other days will be spent deleting unnecessary code, and occasionally, a software-breaking bug will be fixed by changing a single character. Lines of code are only loosely correlated with productivity. Let me translate that into more familiar terms. Consider your favorite author. If you know that they currently write a respectable 1,000 words per day, you would rightfully worry if they begin writing 100,000 words per day. The care that goes into choosing 1,000 great words can’t be maintained at 100x volume [1]. The problem in prolific AI psychosis is not increased output. (More code and more text may actually be more productive.) The problem lies in the subject’s perception of their output: they can’t assess the quality of their own work. The phenomenon mimics psychosis because the subject experiences a mild disconnection from reality: a defect in critical thinking. Current AI Tools Before I go further, it’s important to describe current AI tools. For most people, AI still means a ChatGPT conversation or a Gemini summary attached to a Google search. While a few people may experience prolific AI psychosis with these tools, state-of-the-art tools pose a much greater threat. There are similarities between current AI tools and chatbots: you enter a prompt; you get a response. But state of the art tools are now wrapped in a program called an agent harness. Instead of answering a simple question through conversation, a harness is designed to work on complex tasks until a solution is found. It acts as an agent by making decisions, opening programs, accessing the internet, and writing software to help you achieve your goals. A great harness is especially impressive in software development. If you wanted to create an iPhone app ten years ago, you’d need to spend dozens of hours reading and writing code to get a basic app working. Today, you can type a few sentences into Claude Code, respond to a few questions, and a new app will pop out a few minutes later. It won’t be great, but it might solve your problem. (Getting it listed on the app store is an entirely different issue that can’t be solved so easily.) A developer can do much more. Harnesses can create loops which repeatedly call AI models, working through a software development todo list until a full-fledged app is built. With discipline, the outcomes are astounding. It feels miraculous when an AI agent completes a difficult task for you. But every superpower has its drawbacks, and AI sometimes makes decisions that are impossibly bad. Multiple new files are created when a one-line fix would have done the job. You’re confidently told that a new solution works, while your actual software is more broken than ever. The code becomes a tangled mess that is difficult to read and almost impossible to extend. Simple, decades-old problems that have been solved by open-source solutions are rebuilt from scratch, making your product painfully slow. We’ve found ways to mitigate some of these issues, but it’s a skill that takes time to develop. Some claim that current AI models work at the level of a graduate student. That doesn’t match my experience. They’re one part senior engineer and one part toddler-running-across-white-carpet-with-a-jug-of-red-Kool-Aid. Your job is to determine which is which. Unfortunately, both sides speak with complete confidence in their abilities, and they can’t always tell when they’ve spilled the Kool-Aid. And so the software engineer’s job now includes a large amount of difficult supervision [2]. Here’s a different way to think about it: AI software development feels like playing the world’s most favorable slot machine. Most pulls on the lever are big wins! Most of the losses are obvious and small. Occasionally, a loss will look just like a win. And unless you have the skill, focus, and patience to reject counterfeit wins, your mistakes will eventually create chaos. The Development of Prolific AI Psychosis To illustrate the progression of LLM-tool use into prolific AI psychosis, consider a hypothetical example. Imagine that you are a software engineer who now has a tool which can easily complete some tasks 100x faster than ever [3]. If you like building software, this is beyond exciting. You can accomplish more at work. You can build the hobby project you’ve always wanted. You can finally create the video game of your dreams. Every possibility is on the table. And so you pull the slot machine lever and get started. While the wheels spin, you set up more machines. You rush back to review results as they come. They’re good this time. But you keep catching occasional bugs. And so you set up new machines to help you check your work. Sometimes, they catch problems, and sometimes they don’t. So you refine their instructions. You give them more capabilities. You create networks of bots which work together to solve problems. The code keeps coming. Your ambitions grow. You’re not just creating new software; you’re going to be the first developer to run a one-billion-dollar company. It’s been predicted. Why not you? The scope of your project increases. You can’t sleep at night because you’re so excited to work. You spend more and more time with computers, and less and less time with people. Those who question your efforts don’t understand. Their criticisms are short-sighted. And even if the models aren’t perfect now, the next set of tools will be better. Over time, your lack of sleep and hyperfocus draw you away from reality. Your software is impressive in many ways. But is it useful? Is it good? Is it something that people actually want? You don’t have time for these questions. You need to “escape the permanent underclass.” I feel stressed just thinking about it. While I don’t think I’ve ever experienced full-on AI psychosis, I have been through brief periods where I was headed down this path. It feels incredible to make rapid progress. And there are times when the model’s output is nothing short of miraculous. The illusion broke when I realized that I couldn’t understand my own project. That’s not an unfamiliar feeling in tech: every advance is built upon layers of progress that I’ll never fully appreciate [4]. But I had never before written dozens of custom files without the faintest idea of how they worked. And it became impossible for me to add new features without starting over from scratch. I’ve spoken with many developers who’ve been through a similar experience. Life is much better on the other side. There are dozens of productive AI workflows—each with advantages and disadvantages. But the most effective developers all seem to prioritize human judgment, sleep, and at least some semblance of a life outside of work. The Cause of Prolific AI Psychosis I can only speculate on the cause of prolific AI psychosis. The slot-machine-style rewards of AI tools are partially responsible. Intermittent reinforcement—the receipt of positive or negative outcomes on an unpredictable schedule—is one of the most powerful drivers of human behavior. It’s also an important feature of LLM technology that can’t be wholly removed. Individual factors are also important. I suspect that people with ADHD and impulse-control problems are at increased risk of prolific AI psychosis. Likewise, personality traits like high openness and low conscientiousness may correlate with risk. Experience may play a role. It’s not yet clear whether new developers or senior engineers are more at risk. I suspect that new programmers are more susceptible, but accounts of prominent developers experiencing apparent prolific AI psychosis are relatively common [5]. Cultural excitement also influences developer behavior. These tools are powerful when used well. But every technological revolution is accompanied by unwarranted hype and get-rich-quick schemes. It’s easy to ignore downsides when the upside is so appealing. The narrative that AI will replace all jobs doesn’t help [6]. This message has softened in recent months, but the fear remains significant. This is especially true for software developers, who often find it difficult to find a new role after layoffs. More output—even when it is not truly productive—still drives rewards in metrics-driven environments. Finally, “taste” is increasingly relevant. Unfortunately, it’s also difficult to define. I prefer the term “craft,” which to me encompasses both quality and aesthetics. Regardless of language, I believe that product assessment is a conversation between humans. Is it useful? Is it good? Is it desirable? Is it delightful? These are subjective qualities that are difficult to find within an LLM [7]. I’ll close by adding that prolific AI psychosis isn’t just a problem for software developers. Anything that can be automated with AI can turn to slop with insufficient human guidance. Placing intentional boundaries is one of the most important tasks of the AI era. That’s enough for today. Next week, I’ll write about parasocial relationships with LLM chatbots. Footnotes: [1] This comparison is imperfect. Software and novels are fundamentally different. Every software project has boilerplate code and a limited vocabulary. Likewise, the problems that arise in software development are different in kind from the problems seen in other creative endeavors. [2] Much of the recent work that has advanced agentic engineering is an attempt to address this challenge. But the best work still requires a significant amount of human supervision. [3] I’m skeptical of claims that AI makes people 100x more effective at work. I have found instances where a few minutes with a model can save a day’s worth of work, which matches the 100-fold order of magnitude. But the average increase in productivity is not this significant. Every moment saved also includes an increase in AI supervision; this is a non-trivial task. In addition, there are other bottlenecks to productivity. These include coordination, experimentation, understanding of the customer, access to resources, and internal policies. [4] This comic is legendary for a reason: https://xkcd.com/2347/. [5] Anecdotes are not data, and so further research will be necessary. Likewise, accusations of “AI psychosis” are not always merited. “This doesn’t meet my personal standard of quality” is a criticism that predates generative AI. [6] Cultural beliefs that are shared within a community are not delusional. Tech communities have their share of quirks. It is not a delusion to believe (or disbelieve) in “a permanent underclass,” artificial superintelligence, or the existence of an AI bubble. [7] I’m aware that people are working on agentic economies which sell directly to AI agents. It’s not an area I’m interested in, but it does raise questions about non-human taste.