What Anthropic’s latest AI discovery does—and doesn’t—show


One niche that Anthropic spends more time and money on than other AI companies is called mechanistic interpretability, which means looking inside the complex math of an AI model to learn why it comes up with one particular output and not another. It’s complicated stuff; there are millions of data points that might contribute to any result, and wading through them can look more like word salad than anything useful. It’s also controversial. Describing AI models with terms borrowed from psychology and neuroscience can make their behavior seem more sophisticated than we might otherwise judge it to be.

That’s why, when Anthropic announced last week that it had found a new window into its models’ “internal thoughts” as they reason through answers, there was one colleague I had to talk to. Senior editor Will Douglas Heaven, aside from having a PhD in computer science, has spent a lot of time digging into what we can say about how AI models work. I spoke with him about what we should take from Anthropic’s new (and predictably quirky) research.

What did Anthropic learn here, exactly?

Anthropic has been trying to understand how large language models (LLMs) work for a few years now. Anthropic isn’t the only one looking at this, but I think the company has made it part of its core mission more than most. Anthropic’s CEO, Dario Amodei, has said we won’t be able to control LLMs fully unless we learn more about how they work. 

So this new research is very much in that context. It goes deeper into the weird mechanisms inside LLMs than ever before. What Anthropic learned was that LLMs have a space inside them—which Anthropic calls the J-space—filled with words that don’t appear in their output but that seem to influence the way they puzzle through problems. All this was hidden until Anthropic developed a new technique to probe its model Claude, so it’s a genuine discovery. 

Sometimes these words keep track of where the LLM has got to in a particular task, sometimes they look more like flashes of recognition (for example, “protein” might pop up when you give an LLM only the letters of a protein sequence), and sometimes they represent a kind of internal commentary on the model’s decision-making. In my favorite example, Claude decided to cheat on a coding test when the word “panic” appeared.

Anthropic also found that LLMs are able to describe and manipulate the words in this space. So somehow they seem to be making use of it. 

Let’s step back for a second. I don’t think of large language models as simple, but they’re also not magic. There’s a bunch of math that learns relationships between words, right? So why is it so hard to “peer” into an LLM to know what’s going on?



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


TL;DR

India debates sovereign AI after the US forced Anthropic to kill Fable 5, with proposals for a $5B fund and calls to embrace open-source models.

When the US government ordered Anthropic to shut down Fable 5 and Mythos 5 on 12 June, the export control directive was aimed at restricting foreign nationals from accessing America’s most capable AI. In India, Anthropic’s second-largest market, it landed as a warning shot about what happens when your AI infrastructure runs on someone else’s politics.

The suspension cut off Indian developers and enterprises from Claude’s most advanced models overnight. India’s Claude run-rate revenue had doubled since October 2025, and Tata Consultancy Services had announced a partnership just one day earlier, on 11 June, to train 50,000 employees on Claude and build a dedicated Anthropic business unit. That deal is now in limbo.

The timing has turned what was already a simmering debate about AI sovereignty into a full strategic reckoning. Proposals that sounded ambitious a week ago now sound urgent.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol’ founder Boris, and some questionable AI art. It’s free, every week, in your inbox. Sign up now!

Mohandas Pai, former Infosys CFO and one of India’s most prominent tech investors, has called for a ₹50,000 crore (roughly $5 billion) annual sovereign AI fund. He has also proposed a ₹2 lakh crore (approximately $21 billion) credit guarantee to finance cloud infrastructure, hardware procurement, and semiconductor development. The figures dwarf the government’s existing commitment.

India approved its IndiaAI Mission in March 2024 with a budget of ₹10,372 crore, approximately $1.25 billion. The programme has deployed around 38,000 GPUs so far. Pai’s proposal would quadruple annual spending and add a credit backstop an order of magnitude larger.

Sridhar Vembu, the founder of Zoho, has gone further. He argued that India should embrace smaller and open-source models, including Chinese ones, rather than depend on American frontier systems that can be switched off by executive order. “Technology is the ultimate weapon,” Vembu said. “Globalization is dead and Bharat must find her own way ahead.

The argument has teeth because the suspension demonstrated exactly the vulnerability Vembu is describing. Amazon’s CEO reportedly triggered the government crackdown by telling Treasury Secretary Scott Bessent that researchers had used Fable 5 to obtain information that could be used in cyberattacks. Anthropic called the action disproportionate, but compliance was immediate and global.

Policy expert Prasanto Roy put it bluntly: “American AI models are bound to American geopolitics.” For Indian enterprises that had built workflows around Claude, the lesson was that access to frontier AI is a privilege that can be revoked without notice, without consultation, and without regard for the commercial relationships it disrupts.

The Indian startup ecosystem is already adapting. Sarvam, a Bengaluru-based AI company, released 30-billion and 105-billion parameter open-source models at the India AI Impact Summit in 2026. Krutrim, founded by Ola’s Bhavish Aggarwal, has pivoted from building foundational models to providing cloud and AI infrastructure services, reporting ₹3 billion in revenue for fiscal year 2026.

Neither company is close to matching the capabilities of Fable 5 or Mythos 5. But the argument for sovereign AI was never about matching frontier performance immediately. It is about ensuring that the floor does not fall out when Washington makes a unilateral decision about who gets to use which models.

Aakrit Vaish, founder of the AI startup Activate, said the suspension “completely changes things” for the sovereign AI debate. Vijay Rayapati, CEO of Atomicwork, raised concerns about what the precedent means for Indian companies with multi-country teams that depend on American AI providers. If the US can shut off model access to enforce export controls, any country that relies on American AI is one policy decision away from disruption.

Not everyone agrees that India needs to build its own frontier models. Hemant Mohapatra, a partner at Lightspeed Venture Partners, argued that talent and compute access matter more than capital for building competitive AI. India has the engineering workforce, but the compute gap is significant, and closing it requires either massive domestic investment or continued access to foreign cloud infrastructure.

Anthropic opened a Bengaluru office as part of its India expansion, and the TCS partnership was designed to be a cornerstone of its enterprise strategy in the country. Whether those plans survive the suspension intact depends on how quickly Anthropic can restore access and whether Indian enterprises still trust a provider whose most capable models can vanish overnight.

The broader pattern is unmistakable. The US has spent four years tightening controls on AI technology, from chip export restrictions to model-level interventions. Each escalation pushes more countries toward the conclusion that dependence on American AI infrastructure carries political risk. India, with its 1.4 billion people and rapidly growing technology sector, is now asking whether it can afford that risk, and what it would cost to eliminate it.

The Opendoor layoffs in June 2026, which shut the company’s India office and affected roughly 250 employees, added another dimension. CEO Kaz Nejatian cited AI-native teams as the reason, suggesting that some US companies are using AI to reduce their reliance on Indian engineering talent at the same time that India is debating its reliance on American AI. The relationship is becoming less complementary and more competitive.

For now, the sovereign AI proposals remain proposals. Pai’s fund has no legislative vehicle, Vembu’s call for open-source adoption has no coordinated policy framework, and the IndiaAI Mission’s GPU deployment is still in early stages.

But the Anthropic suspension has done something that years of policy papers and conference speeches could not: it has given the sovereign AI movement a concrete, recent, and viscerally felt example of why dependence on foreign AI is a strategic liability. The debate is no longer theoretical.



Source link