Bug Blindness

someone told me the results didn't reproduce on Google when they tried it some number of weeks later. Of course it didn't, which I discussed here in more detail, but for the short of it, here's this post about scams and other bad results on Google that was #1 on HN for a while. Of course somebody fixed that! And, also, ad results are non-deterministic and, while there are a lot of bad ads, it's not like the majority are scams, so you wouldn't expect to get scam ads at the top results even if someone else did for the same query. [return] For example, anyone familiar with my code at Twitter will recall the huge comments I had at the top of the main files for the things I owned, which described the various ways in which the thing is really flawed. They were all things that, for one reason or another, I thought weren't worth the time to fix, but they were still serious problems that anyone interacting with the code ought to know about. For this metrics project, I even had a long doc that described the issues in great detail (IIRC, in a lot of cases, the rough shape of the fix was described; maybe today an LLM could take that and fix it). I have the same feeling about my writing. While a huge number of bugs sneak through my writing (like spelling and grammatical errors), most of those are things I sort of don't care about and will skim past in other people's writing as well. When I say don't care, it's not that I don't want things to be better (when people send me corrections I generally fix things), it's just that my brain doesn't naturally pay attention to those things no matter whose writing it is, so I don't seem to have a particular blind spot in my writing with respect to these kinds of bugs. For the things I do care about, I could edit posts endlessly because, no matter how much I edit, the post still seems pretty bad to me. I used to often (and still sometimes) send a post to someone and ask them if it makes any sense to publish it at all (since I don't like it due to all of the issues I see). There are various tricks I've used to get around this (not explicitly to get around this, but they do so as a side effect). As discussed in this old post on writing, for a while, I hired a professional editor and had a process goal of doing one pass on each post and then trying to improve the next post. And as noted in the postscript to recent posts, now I'm trying to write with extremely minimal cleanup and editing and push posts out in half an hour regardless of the state of the data I'm looking at or the post (which I'm generally failing to do; I thought I might succeed on this one because it doesn't have any data analysis, but someone made a comment on the draft post that got me to re-write the whole thing, and just on number of words in the post, half an hour would really be pushing it on the original and then it increased in length). Of course a post that's written as quickly as possible with little to no regard for cleaning things up is going to be terrible in all kinds of ways, so all flaws I see in the post don't stop me from publishing it. [return] For example, anyone familiar with my code at Twitter will recall the huge comments I had at the top of the main files for the things I owned, which described the various ways in which the thing is really flawed. They were all things that, for one reason or another, I thought weren't worth the time to fix, but they were still serious problems that anyone interacting with the code ought to know about. For this metrics project, I even had a long doc that described the issues in great detail (IIRC, in a lot of cases, the rough shape of the fix was described; maybe today an LLM could take that and fix it). I have the same feeling about my writing. While a huge number of bugs sneak through my writing (like spelling and grammatical errors), most of those are things I sort of don't care about and will skim past in other people's writing as well. When I say don't care, it's not that I don't want things to be better (when people send me corrections I generally fix things), it's just that my brain doesn't naturally pay attention to those things no matter whose writing it is, so I don't seem to have a particular blind spot in my writing with respect to these kinds of bugs. For the things I do care about, I could edit posts endlessly because, no matter how much I edit, the post still seems pretty bad to me. I used to often (and still sometimes) send a post to someone and ask them if it makes any sense to publish it at all (since I don't like it due to all of the issues I see). There are various tricks I've used to get around this (not explicitly to get around this, but they do so as a side effect). As discussed in this old post on writing, for a while, I hired a professional editor and had a process goal of doing one pass on each post and then trying to improve the next post. And as noted in the postscript to recent posts, now I'm trying to write with extremely minimal cleanup and editing and push posts out in half an hour regardless of the state of the data I'm looking at or the post (which I'm generally failing to do; I thought I might succeed on this one because it doesn't have any data analysis, but someone made a comment on the draft post that got me to re-write the whole thing, and just on number of words in the post, half an hour would really be pushing it on the original and then it increased in length). Of course a post that's written as quickly as possible with little to no regard for cleaning things up is going to be terrible in all kinds of ways, so all flaws I see in the post don't stop me from publishing it. This was, inadvertently, a kind of revenge on my friend for when I tried to open his door for the first time to leave his place. Since the door clearly opened to the outside, I tried pushing on the door, which didn't work, so I checked if there was a latch that was stuck, if the door was still locked, if I needed to push harder, etc., none of which worked. When he saw that I couldn't open the door I asked him what the trick was he said, in a tone of voice that made it sound like this was obviously something everyone should know, you need to pull the door before pushing it. The door was wedged such that the easiest way to open the door was to pull the door as tightly shut as possible and then immediately shove the door open. This friend, since he grew up in that house, thought this was obvious, apparently not realizing that it's not normal to have to try to close a door extra hard to open it. [return] a response I've heard to this kind of thing recently is that Anthropic had the best growth numbers in history while Claude was very buggy. If you have the best coding model and agent in the world, you can get away with a lot, but even they seem to have spent a fair amount of effort improving quality. Maybe you can also get away with it if you have a product that succeeds due to bundling, the strength of your enterprise sales team, network effects, monopoly power, etc.; all but one of the cases I'm thinking of are places where the team didn't have these things on their side. I actually thought the one other case I was thinking of would be something like Blackboard, but (if the Google results are accurate) I see that the software has declined from being #1 in the market to being a minority player, so maybe they couldn't get away with it either (I didn't look into the reasons for the decline; perhaps it's a coincidence). As noted above, Blackboard is an example where you could argue that the software quality didn't matter and people might as well just believe whatever makes them happy; if thinking that users love the software, then why not think that? But most of the rest of the examples that come to mind for me aren't cases like that. I don't think this is the best example, but it comes to mind because the comment below is the last time I was reminded of the Blackboard example. There was a comment from a Tumblr employee who said that they'd solved the moderation (abuse / spam / toxicity / etc.) problem mechanically at Tumblr via the way reblogs worked and that the mechanics Tumblr provided to users were good enough that the community could self-police bad behavior and that other social media sites would do well to learn from Tumblr. This was referring to Tumblr back in its heyday (maybe 2009-2014). I never really read much on Tumblr so I don't personally have an opinion, but back when it was a major social media platform, the reputation among folks I know was that it was heavy on bad behavior, particularly pile-ons caused by people taking out of context quotes and turning them into ragebait (not to say this doesn't happen on other platforms, but the belief was that the way Tumblr was structured and/or the communities involved made this worse on Tumblr). I'm not sure I know anyone who used Tumblr at the time who would say that the community was good at self-policing. In fact, when Scott Alexander wrote one of his most famous pieces, Toxoplasma Of Rage, he dedicated an entire section to how Tumblr's reblog system is particularly bad and is guaranteed to result in bad behavior. He actually says that whoever designed the system either didn't understand what they were doing or they understood all too well and deliberately made the most ragebait-inducing system possible. This was written during the time when this employee said that Tumblr had solved the moderation problem and uses examples from that time. Moderation at scale is an impossibly hard problem, so as a non-Tumblr user, I'm not even sure that Tumblr did worse than other platforms given its size and growth rate, but I think you'd need some quality blindness to think that Tumblr had solved the moderation problem. I think the strongest positive case you could plausibly make would be something like "Tumblr was better than average, but many people had a worse than average experience due to the communities they were in and some of these communities were unusually widely read and Tumblr therefore unfairly gained a reputation as being a particularly bad platform". I don't know if that's true or not, but it doesn't seem impossible that it could be true; it does seem impossible that Tumblr solved the moderation problem. [return] a response I've heard to this kind of thing recently is that Anthropic had the best growth numbers in history while Claude was very buggy. If you have the best coding model and agent in the world, you can get away with a lot, but even they seem to have spent a fair amount of effort improving quality. Maybe you can also get away with it if you have a product that succeeds due to bundling, the strength of your enterprise sales team, network effects, monopoly power, etc.; all but one of the cases I'm thinking of are places where the team didn't have these things on their side. I actually thought the one other case I was thinking of would be something like Blackboard, but (if the Google results are accurate) I see that the software has declined from being #1 in the market to being a minority player, so maybe they couldn't get away with it either (I didn't look into the reasons for the decline; perhaps it's a coincidence). As noted above, Blackboard is an example where you could argue that the software quality didn't matter and people might as well just believe whatever makes them happy; if thinking that users love the software, then why not think that? But most of the rest of the examples that come to mind for me aren't cases like that. I don't think this is the best example, but it comes to mind because the comment below is the last time I was reminded of the Blackboard example. There was a comment from a Tumblr employee who said that they'd solved the moderation (abuse / spam / toxicity / etc.) problem mechanically at Tumblr via the way reblogs worked and that the mechanics Tumblr provided to users were good enough that the community could self-police bad behavior and that other social media sites would do well to learn from Tumblr. This was referring to Tumblr back in its heyday (maybe 2009-2014). I never really read much on Tumblr so I don't personally have an opinion, but back when it was a major social media platform, the reputation among folks I know was that it was heavy on bad behavior, particularly pile-ons caused by people taking out of context quotes and turning them into ragebait (not to say this doesn't happen on other platforms, but the belief was that the way Tumblr was structured and/or the communities involved made this worse on Tumblr). I'm not sure I know anyone who used Tumblr at the time who would say that the community was good at self-policing. In fact, when Scott Alexander wrote one of his most famous pieces, Toxoplasma Of Rage, he dedicated an entire section to how Tumblr's reblog system is particularly bad and is guaranteed to result in bad behavior. He actually says that whoever designed the system either didn't understand what they were doing or they understood all too well and deliberately made the most ragebait-inducing system possible. This was written during the time when this employee said that Tumblr had solved the moderation problem and uses examples from that time. Moderation at scale is an impossibly hard problem, so as a non-Tumblr user, I'm not even sure that Tumblr did worse than other platforms given its size and growth rate, but I think you'd need some quality blindness to think that Tumblr had solved the moderation problem. I think the strongest positive case you could plausibly make would be something like "Tumblr was better than average, but many people had a worse than average experience due to the communities they were in and some of these communities were unusually widely read and Tumblr therefore unfairly gained a reputation as being a particularly bad platform". I don't know if that's true or not, but it doesn't seem impossible that it could be true; it does seem impossible that Tumblr solved the moderation problem. most of my projects are deliberately low quality; what I try to do is do the highest ROI testing, not test to the point the quality is what I would actually consider good, and this also goes for things like making interfaces very nice, etc. [return]