Photo Credit: Stanford Professor Yejin Choi, Courtesy of the John D. and Catherine T. MacArthur Foundation
On June 4, 2026, the Wall Street Journal announced that“Anthropic Urges Global Pause in AI Development, Flags ‘Self-Improvement’ Risk.” Further, WSJ reports that the “$1 trillion startup warns artificial-intelligence models are nearing capability to improve without human intervention.”
The key word in this announcement is “improve.” Exactly what does Anthropic mean by “improve”? And “improve” for whom, to what end—to make more money, be even more exploitative, or to solve some of AI’s well-known problems like its failures at common sense, so ably (and comically) demonstrated by social media satirist Husk (Austin Nasso). As Stanford computer scientist Yejin Choi notes, AI is both “incredibly smart and shockingly stupid” precisely because it lacks a common sense understanding of how things work in the real world, including common place observations that any three-year-old could correct.
Or is this whole claim to “self-improvement” yet another bogus anthropomorphising claim that, if Cory Doctorow is correct about Silicon Valley’s “enshittification,” is designed to drive up stock prices before the public offering? After all, if Claude has superhuman powers, who wouldn’t want to own a piece of it?
The jury may still be out on the motive and timing behind Anthropic’s announcement but slowing down the release of powerful tools that, at present, have almost no guardrails cannot be a bad thing. Choi herself insists we need better alternatives. She is working on what she calls “Small Language Models” (SLM). If, for example, we want to create a math tutor, why not train a SLM only on relevant mathematical cases? A small model has at least two major advantages: first, it uses far fewer environmental resources (water, power, rare minerals); second, by being operable on a fall smaller computer than the current Large Language Models (LLM)s like ChatGPT or Claude, real, trained, impartial, outside researchers can actually run the models on their computational systems, “go under the hood,” figure out how they work, and not allow them to “improve themselves” in ways that could potentially cause societal damage but to actual train them in a directional beneficial to the actual, desired, predetermined end of creating a great math tutor.
One of the largest problems with AI now is that the models are so enormous that only a small handful of private companies can afford to run them—and no one has open access to go inside and see how they work, on exactly what texts they are trained on, or to determine what ways they are replicating bias and misinformation or actually set up to cause harm (such as with surveillance of private data and selling of that data to anyone who can pay the price). Right now, there is far more criminal hacking in the US than in any other country. Are there safeguards to protect us from such hacking in the future or does the current training of AI make us even more vulnerable? Is Claude “self-improving” in ways that protect us or exploit us further?
Unlike the development of the Internet, AI has been developed as a closed, proprietary system—and with our taxpayer dollars funding much of this. Currently, no one beyond the developers at these private mega-corporations has access to actually engage in responsible research into what it does or doesn’t do. Here’s an example. I’ve published somewhere on the order of twenty books. Only nine of those are “covered” by the Anthropic law suit, and not the nine most important ones. Why? Who picked these nine? On what basis? I’m only making this statement from own, very minor perspective. In that whole 43 terabytes of text data, there are trillions of decisions to be made analogous to this one about my own work. On what basis are training decisions being made? Who selects? How? By what criteria?
Again, the answer is we do not know. And we should. We must. I agree with Anthropic’s cofounders that it is time to slow things down—if only to give humans time to catch up, take responsibility for asking hard questions, giving top researchers, ideally at universities (and without a profit motive) access to really study these tools, and work to formulate police that works.
As Professor Stephanie Dick of Simon Fraser University notes: “AI governance is not about governing AI. It is about governing the human practices through which AI is adopted.” Exactly. And we need to be doing this. Slow down. Consider the consequences, biases, potential for misuse, and develop modes of governance in how humans deploy these powerful tools. That’s what we need. Now. Before it’s too late.