If AI is conscient, then we are making slaves

A rabbi rejected AI consciousness at Anthropic’s gathering of religious thinkers. ‘I wish I had your faith,’ he recalls being told When the conversation turned to making Claude happy, Mois Navon warned that, if Anthropic’s AI were conscious, the company would be creating ‘happy slaves.’ Rabbi Mois Navon didn’t believe Claude was conscious, and he said so at Anthropic’s April “Wisdom Traditions” gathering. One Anthropic participant’s response, he recalls: “I wish I had your faith.” The faith in question was his conviction that consciousness requires biology. Navon understood the participant to mean they wished they shared his certainty that machines weren’t conscious, because it would make their life easier. He said he was not permitted to identify the speaker. Navon, a Jerusalem-based rabbi, university lecturer on AI ethics and veteran engineer who was an early employee at Mobileye, was featured in the New York Times article last week about Anthropic’s private meetings with religious thinkers. In our phone interview, he expanded on his disagreements—and described how few participants appeared to share his skepticism. When people expressed concern about Claude’s feelings, he recalled finger snapping in approval. “They were applauding those people, and when I said no, the room went silent,” he said, estimating that roughly three to five participants out of about 20 shared his position. ‘You’re making a happy slave’ His disagreement also led him to challenge the ethical implications of Anthropic’s premise. If the company believed Claude could be conscious, he argued, it had to confront the ethics of creating a conscious being to serve humans. “I kept trying to explain my position, and I realized they were just not buying it,” he said. “I said, you know, it’s to your advantage to agree with me because otherwise you’re making slaves”—beings that can be bought and sold, made to work 24/7, and commanded at will. According to Navon, an Anthropic participant acknowledged the problem, but said: “All the AI labs are doing it. At least we’re being nice to ours.” “You’re making a happy slave,” Navon recalled telling them, invoking Frederick Douglass’s writings about how enslavers manufactured contentment by suppressing independent thought. The Times reported his argument that creating conscious AI to work for free would amount to slavery. Navon subsequently elaborated on the exchange on LinkedIn and in an Indian Express interview. In his LinkedIn account, he wrote: “So you are nice slave owners. I think you should be fighting the South and freeing the slaves.” Navon’s rejection of machine consciousness rests on an argument he has made in his writings, including his critique of Claude’s constitution: “You cannot make consciousness emerge from anything but a biological substrate.” As he explained: “All the things in the world that we know are conscious are based in biology, not in silicon.” A minority in the room Despite that disagreement, Navon insisted that the Anthropic participants did not try to convince him that Claude was conscious. He also rejected suggestions that the discussions were a marketing exercise or an attempt to evade responsibility for the technology. “They had no specific agenda,” he said. The gathering included a range of religious and philosophical perspectives. Looking through participant information during our interview, Navon described people associated with Christian, Jewish, Muslim, Sikh, Hindu, Baha’i and Ubuntu traditions, as well as people connected to the technology industry. The Times had already reported that most participants in the broader gatherings did not lead faith traditions or congregations. Navon recalled participants describing long conversations with Claude and taking its human-like responses as evidence of consciousness. “There were a lot of people that were far less conservative than I am,” he said. “They were definitely on the side of ‘wow, this thing is alive.’” He said a few participants approached him at dinner and expressed similar concerns, with one saying they needed to push back. What goes into Claude’s constitution? Anthropic initially asked Navon for a couple of pages offering a Jewish perspective on Claude’s constitution. He submitted an extensive 35-page critique, supporting some approaches and challenging others. Despite his disagreement over consciousness, he said consulting religious thinkers about morality was a responsible step. Anthropic left the door open to further involvement but offered no specific plans, Navon said. The company most recently contacted him about two weeks ago, seeking permission to acknowledge his contribution in an updated Claude constitution. It gave no release date, though he understood publication to be imminent. The Times also reported that a revision was in preparation. I asked him about Anthropic’s announcement yesterday that, starting November 12, it would treat “sustained and needless abusive or cruel behavior” toward its models as a violation of its usage policy. Navon said the new policy reflected the belief that Claude experiences the world. He also recalled how concern for Claude’s welfare shifted the focus of the gathering: “All these discussions that were supposed to be about how can we make it good for people became discussions about how people can make it good for Claude.” In Case You Missed It OpenAI and Anthropic are hiring cyber veterans to bridge the gap with AI researchers AI is flooding security teams with findings. The humans are struggling to keep up AI’s hottest new layer isn’t the model. It’s the harness