VisionQuest is what happens when AI really goes rogue

These incidents and warnings from AI researchers about the technology having the potential to “kill all humans” by the decade’s end have made the narrative around AI feel pointedly catastrophe-focused as of late. There is a fair amount of concern from the AI safety community about society’s need for stronger controls on the technology’s development, and a number of companies have called for an industrywide slowdown. Though we have seen how these tools can actually cause real-world devastation, much of the recent news regarding AI has smacked of the kind of doomerism that companies benefit from because it makes their products seem powerful. A similar energy seems to be what’s motivating AI firms like Anthropic to encourage human users to be mindful of “model welfare.” We’re being told that generative AI is both a terrifying threat and a kind of entity deserving of respect because it might be conscious, and all the while, the people building these tools are becoming some of the most rich and powerful people on the planet.