How accurate have Ed Zitron's AI skeptic predictions been?

But, even if it were the case that the accusations against Raghavan are true (I'm not sure how they could be, as how could one be a class traitor to computer scientists in the first place, but let's posit that, whatever it means, it's true), Zitron's contention is that "this shithead [points to an image of Raghavan] took over Google search in 2020" and then prioritized certain metrics over search quality. I'm not sure why one would name a particular person for this as this is something that was a long-standing fight with many people involved on all sides but, if we posit that this is all true, then we posit that the "management consultancy sect" will move metrics that will cause engagement and/or revenue to increase at the cost of search quality. This would have the opposite of the effect Zitron needs here to make his case that Google growth is done and they're so desperate for growth they have to put AI everywhere in some kind of crazed last-ditch attempt to save Google. Perhaps one could make the argument that this will eventually cause Google search to decline, but Zitron's argument was that, in 2024, they were desperate, not that users will eventually leave Google search, which will later cause a decline. Anyone who's read a lot of Zitron will recognize a standard "move" of his, turning the situation into some kind of hero-villain narrative (for search, the alleged hero is Ben Gomes and the villain is Prabhakar Raghavan); it's as if his mental model of how companies works comes from movies about companies. If you ever watch a movie that's allegedly about some events and then read about it, you'll find that things get oversimplified into a hero-villain narrative and that almost all of the nuance is stripped out of the situation. And then if you're ever personally involved in something or talk to people who are personally involved and compare what happened to the books that get written about it, the same thing happens again; in general, the major causal factors are not identified in books about what happened in tech and many of the most instrumental people involved in some of the key decisions aren't even named because journalists talking to people about what happened aren't really able to piece together a plausibly correct story about what happened to someone who understands the underlying mechanics and has good information. Anyway, without knowing anything about the situation, if someone tells you a hero-villain narrative of the kind Zitron likes to spin, you can already be a bit skeptical. [return] But, even if it were the case that the accusations against Raghavan are true (I'm not sure how they could be, as how could one be a class traitor to computer scientists in the first place, but let's posit that, whatever it means, it's true), Zitron's contention is that "this shithead [points to an image of Raghavan] took over Google search in 2020" and then prioritized certain metrics over search quality. I'm not sure why one would name a particular person for this as this is something that was a long-standing fight with many people involved on all sides but, if we posit that this is all true, then we posit that the "management consultancy sect" will move metrics that will cause engagement and/or revenue to increase at the cost of search quality. This would have the opposite of the effect Zitron needs here to make his case that Google growth is done and they're so desperate for growth they have to put AI everywhere in some kind of crazed last-ditch attempt to save Google. Perhaps one could make the argument that this will eventually cause Google search to decline, but Zitron's argument was that, in 2024, they were desperate, not that users will eventually leave Google search, which will later cause a decline. Anyone who's read a lot of Zitron will recognize a standard "move" of his, turning the situation into some kind of hero-villain narrative (for search, the alleged hero is Ben Gomes and the villain is Prabhakar Raghavan); it's as if his mental model of how companies works comes from movies about companies. If you ever watch a movie that's allegedly about some events and then read about it, you'll find that things get oversimplified into a hero-villain narrative and that almost all of the nuance is stripped out of the situation. And then if you're ever personally involved in something or talk to people who are personally involved and compare what happened to the books that get written about it, the same thing happens again; in general, the major causal factors are not identified in books about what happened in tech and many of the most instrumental people involved in some of the key decisions aren't even named because journalists talking to people about what happened aren't really able to piece together a plausibly correct story about what happened to someone who understands the underlying mechanics and has good information. Anyway, without knowing anything about the situation, if someone tells you a hero-villain narrative of the kind Zitron likes to spin, you can already be a bit skeptical. BTW, I don't think his anger really comes across in the video. I mean, he explicitly says he's angry and he swears and insults people, just like in his writing, but he doesn't really read as angry to me. It reminds me of this test on emotion recognition I took with a bunch of folks recently. I found the test fairly difficult and spent maybe 5 minutes on the first question because the person had a huge fake smile on their face and also looked a bit uncomfortable and anxious. I couldn't tell if you were supposed to say that the person is happy or uncomfortable/anxious. Is it supposed to be a very easy test or is it supposed to be a test that has a bit of subtlety? Based on what the test looked like, after thinking about it for a while, I chose "happy". Luckily, the test actually tells you if you got the question right or not, so I realized the test was about the fake exaggerated expression being made and not the person's actual expression and most the rest of the questions were easy (one was difficult because they were faking one particular emotion with what is a textbook display, as in, the kind of thing one sees in a textbook, but in a very specific way that was less complete and more unrealistic than the other textbook displays). Anyway, to me, Zitron seems like someone who's playacting anger and not someone who's actually angry. The tone of voice, facial expression, body language, style of movement, etc., just don't seem angry to me. I think this anger positioning works better in his writing than in his speeches because the cues he uses (swearing, saying he's angry, showing a lot of contempt, insulting people, etc.) are about as good as it gets for anger cues in writing. When you have audio and video, these are fairly weak cues; if the stronger cues don't really indicate anger, the person just doesn't seem angry. [return] BTW, I don't think his anger really comes across in the video. I mean, he explicitly says he's angry and he swears and insults people, just like in his writing, but he doesn't really read as angry to me. It reminds me of this test on emotion recognition I took with a bunch of folks recently. I found the test fairly difficult and spent maybe 5 minutes on the first question because the person had a huge fake smile on their face and also looked a bit uncomfortable and anxious. I couldn't tell if you were supposed to say that the person is happy or uncomfortable/anxious. Is it supposed to be a very easy test or is it supposed to be a test that has a bit of subtlety? Based on what the test looked like, after thinking about it for a while, I chose "happy". Luckily, the test actually tells you if you got the question right or not, so I realized the test was about the fake exaggerated expression being made and not the person's actual expression and most the rest of the questions were easy (one was difficult because they were faking one particular emotion with what is a textbook display, as in, the kind of thing one sees in a textbook, but in a very specific way that was less complete and more unrealistic than the other textbook displays). Anyway, to me, Zitron seems like someone who's playacting anger and not someone who's actually angry. The tone of voice, facial expression, body language, style of movement, etc., just don't seem angry to me. I think this anger positioning works better in his writing than in his speeches because the cues he uses (swearing, saying he's angry, showing a lot of contempt, insulting people, etc.) are about as good as it gets for anger cues in writing. When you have audio and video, these are fairly weak cues; if the stronger cues don't really indicate anger, the person just doesn't seem angry. If you want to see an example of what it looks like when someone tries to discuss the numbers, here's a thread where Juho Snellman pushes back on someone who insists that people have done the math. As I've been catching up AI discussions, I've seen many discussions like this where one side has someone who's actually looked at the numbers and the other side waves around some kind of vague insistence that numbers have been looked at. This never really goes anywhere because, for one of the sides, the point isn't that you can understand something from the numbers, it's that they have a piece of evidence they can wield because someone has looked at the numbers. [return] Zitron's argument at the time was that hallucinations were as good as they were going to get, which meant that AI performance is capped at 2024 levels. Both the overall prediction and the mechanism were wrong. This one seemed wrong at the time, in that I noted here in 2024 that you can make AI code halfway decently by just putting it in a loop and having it run until the code compiles and tests pass; I wasn't a heavy AI user at the time, but anyone who was using AI could see that there were ways that you could mitigate the hallucination rate which weren't being widely applied (this was before coding agents like codex and claude executed code and would check that tests pass, etc.) [return] This is another one that also seemed untrue at the time. I'm not an ML person, but the moment someone told me what an RL environment was, within minutes, I thought of a bunch of ways one could generate synthetic data for improved training. I'm sure none of these were novel and they're things that AI labs are doing; my point is just that anyone who thinks about it for a few minutes can come up with a lot of ways that models could be improved even if there were no new data to find on the internet (not to mention that more effort could be used to get data that isn't just reddit comments or whatever the easiest to scrape content on the internet is). [return] Note that this only scores predictions that have resolved. In that particular post Zitron also states that progress towards AGI will never happen, which is still both fuzzy and difficult to adjudicate and also one that you can never really reliably resolve as positive. Similarly, a prediction in a previous post that some company would have to add subscriptions isn't listed because it's open ended and not really resolvable as a negative (it was implied to have to happen soon, so is arguably wrong, but if one wanted to weasel out of it one could say that it will happen in the future). [return] Here, Zitron also said, "I’ve realized now that it isn’t super useful to attach things to time (though I stand by my prediction) and thus I think it’s more useful to suggest what the terms of the bubble popping actually are". After this point, Zitron makes relatively fewer dated statements after this point and makes many more open-ended unfalsifiable statements. Perhaps a reaction to being wrong so frequently with his previous predictions? [return] In a small piece of optimism, I'll say that this blog seems to have done ok despite not leaning into extremist positions and generally trying to avoid clickbait. This often means that, when I look at some data, I'll see something that looks like it would make for a really interesting viral hit piece, but then on looking more closely, it's actually a boring negative result, like when I ran this quick and dirty programming language eval, which originally appeared to show a very interesting result, which went away once I fixed the obvious eval bugs. Oh well. I'd like it if people published more boring negative results, so I published the boring negative result. I wouldn't be surprised if this blog is within an order of magnitude of traffic as Zitron's substack (server-side stats show 510k uniques in the past month, but who knows how many of those are bots) despite Zitron writing much more frequently than me and pulling out every clickbait trick in the book, while I just occasionally post something when I feel like writing something up. Although my goal obviously isn't to get traffic, if we adjust for the level of time or effort, I don't think this blog does terribly compared to Zitron. Ceteris paribus, I think Zalewski is right on the incentives, and I've seen a lot of people become caricatures of themselves as they lean into what drives the most engagement, but I think doing the opposite can work ok. For example, with a style that could be described as the opposite of clickbait, Simon Willison has written what I suspect is the most widely read blog among programmers for the past 3-4 years (in the same way that, at various times in the past, Joel Spolsky or Jeff Atwood or Steve Yegge seemed to be the most widely read programmer among programmers). Among programmers and other serious users of AI, I would guess that Willison has a larger audience than Zitron. However, it's true that Zitron has a kind of audience that Willison can never really get with his style. In the body of this post, we looked at common defenses of Zitron on forums where people use AI. That was pulled from forums where people use AI. If we look at the world at large, the comments look fairly different. For example, on the video that my friend mentioned, where Zitron repeatedly denies reality and the interviewer pushes back, the top comments at the moment are all in support of Zitron and they also just deny reality and claim that the places where the interviewer pushes back with a piece of reality are the interviewer being biased or just not knowing what he's talking about. Among the top comments, there seems to be little to no engagement with the facts of the matter; it's all mood affiliation. The comments remind me of what supporters say about politicians who use the gish gallop strategy and just say a bunch of outrageous nonsense. I could imagine Zitron running for office one day on the strength of his reality-denying popularity or becoming a demagogue who's a right-hand-man of someone in office, so Zalewski is right in that Zitron's appeal is not one someone is going to get by accurately describing what's happening in AI. But, while I don't know Willison and this could be totally wrong, my impression is that, like me, he's doing something he wants to do anyway and the audience just sort of happened despite him not trying to maximize his audience. When I say it works ok, I mean that he seems to be able to support himself working as a full-time open source developer due to the sponsorships he's gotten (which I would presume are generally because he has such a large audience), which seems like a good outcome even if this doesn't create the kind of mass appeal someone like Zitron can generate. [return] In a small piece of optimism, I'll say that this blog seems to have done ok despite not leaning into extremist positions and generally trying to avoid clickbait. This often means that, when I look at some data, I'll see something that looks like it would make for a really interesting viral hit piece, but then on looking more closely, it's actually a boring negative result, like when I ran this quick and dirty programming language eval, which originally appeared to show a very interesting result, which went away once I fixed the obvious eval bugs. Oh well. I'd like it if people published more boring negative results, so I published the boring negative result. I wouldn't be surprised if this blog is within an order of magnitude of traffic as Zitron's substack (server-side stats show 510k uniques in the past month, but who knows how many of those are bots) despite Zitron writing much more frequently than me and pulling out every clickbait trick in the book, while I just occasionally post something when I feel like writing something up. Although my goal obviously isn't to get traffic, if we adjust for the level of time or effort, I don't think this blog does terribly compared to Zitron. Ceteris paribus, I think Zalewski is right on the incentives, and I've seen a lot of people become caricatures of themselves as they lean into what drives the most engagement, but I think doing the opposite can work ok. For example, with a style that could be described as the opposite of clickbait, Simon Willison has written what I suspect is the most widely read blog among programmers for the past 3-4 years (in the same way that, at various times in the past, Joel Spolsky or Jeff Atwood or Steve Yegge seemed to be the most widely read programmer among programmers). Among programmers and other serious users of AI, I would guess that Willison has a larger audience than Zitron. However, it's true that Zitron has a kind of audience that Willison can never really get with his style. In the body of this post, we looked at common defenses of Zitron on forums where people use AI. That was pulled from forums where people use AI. If we look at the world at large, the comments look fairly different. For example, on the video that my friend mentioned, where Zitron repeatedly denies reality and the interviewer pushes back, the top comments at the moment are all in support of Zitron and they also just deny reality and claim that the places where the interviewer pushes back with a piece of reality are the interviewer being biased or just not knowing what he's talking about. Among the top comments, there seems to be little to no engagement with the facts of the matter; it's all mood affiliation. The comments remind me of what supporters say about politicians who use the gish gallop strategy and just say a bunch of outrageous nonsense. I could imagine Zitron running for office one day on the strength of his reality-denying popularity or becoming a demagogue who's a right-hand-man of someone in office, so Zalewski is right in that Zitron's appeal is not one someone is going to get by accurately describing what's happening in AI. But, while I don't know Willison and this could be totally wrong, my impression is that, like me, he's doing something he wants to do anyway and the audience just sort of happened despite him not trying to maximize his audience. When I say it works ok, I mean that he seems to be able to support himself working as a full-time open source developer due to the sponsorships he's gotten (which I would presume are generally because he has such a large audience), which seems like a good outcome even if this doesn't create the kind of mass appeal someone like Zitron can generate.