Proving the agents are right instead of claiming It



Most AI research tools point a language model at the web and trust the output. Saarth Shah built Sixtyfour around the opposite instinct: grade everything, ship only what improves the score.

Saarth Shah keeps a scoreboard. Every build of Sixtyfour’s research agents is graded against questions a team of experts assembles by hand and checks against real-world cases, vertical by vertical, and the grade is the only thing that decides what ships. The discipline behind that scoreboard is why a payments company will let his software, rather than a human analyst, decide whether a stranger is real.

Most AI research tools start from the same shortcut: point a language model at the open web, ask it to look something up, and let it write. The output reads well. What Shah fixed on early was whether it was right, and how anyone would know.

For a while, the industry blamed hallucination, the models’ habit of inventing facts and sources. That problem is real, and it is slowly getting better. Frontier models hallucinate less than they did a year ago, and pairing them with live web search has narrowed the gap further. But the limit Shah cares about is not the model’s imagination. It is the model’s reach.

A language model with a search box sees what a person with a browser sees, and no more. The facts that decide an investigation usually sit below the surface, in licensed and proprietary records, in unindexed filings, and in the tangled networks of entities that fraud is built to hide within. Surface research cannot reach them.

The models keep getting better, and they still only see what a person with a browser can see,” Shah said. “The answers that matter in this work are a few levels deeper than that, and you have to go and get them.

Roughly 96 percent of enterprises now run AI agents, and the gap between a system that answers a question and one that answers it correctly has become the central problem in production AI. Even the most capable models still struggle on realistic, long-horizon professional work, where a single wrong step derails the result. Shah drew the lesson years before the benchmarks confirmed it: capability and reliability are different properties, and the second has to be engineered on purpose.

What he built is a different shape of system. Sixtyfour’s agents sit between two failing options, the static databases that go stale and the general models that improvise. Each investigation fuses live web research, public records, state and legal filings, open-source intelligence, and licensed data, then compiles it into a report in which every claim points back to its source. Nothing is asserted that cannot be traced to a document, the kind of provenance a court would demand.

In practice, a single query fans out into many. Ask Sixtyfour to verify a seller, and its agents pull corporate filings, cross-reference addresses and officers, scan sanctions and litigation records, search the live web for the business’s real footprint. When a thread leads somewhere, the agent follows it, gathering context until it can settle the question rather than stopping at the first plausible answer. The output is a dossier in which each line links to the underlying document, so a reviewer can audit the machine’s reasoning rather than trusting it.

The part Shah is proudest of is the part most companies skip. For each domain his agents work in, he built evaluation systems that measure their output against known answers, so the company can prove how often it is right rather than assert it in a sales deck. They also do not ship any product improvements unless they show clear improvements on the evaluation system. And each day, they are making their benchmark harder and harder, using real-world samples they see, to keep pushing the frontier.

Every answer has to show its work,” Shah said. “If we cannot point to the filing or the record, we do not ship the answer. The citation is the product, not the decoration.

Building those evals was a large chunk of the work required to be the best in the world, and the hardest part was the truth itself. The existing data vendors cannot serve as the source of truth; their information is too often outdated or thin. So to grade an agent on a genuinely hard case, Sixtyfour sometimes establishes the answer the slow way, running it to ground with human investigators, then turning it into a test the agents must pass. The aim is to match the best investigators, then beat them, until the eval set measures a standard no single reviewer could reach alone.

The discipline is unforgiving. Every new tool the team adds is run against the eval set before it ships, and when a promising addition drags the score down, it comes back out. A rival can copy an architecture diagram in an afternoon. It cannot copy years of graded runs from production systems, and the nuances that push the AI agents to do better.

It is why Sixtyfour sits at the top of the recon benchmark, ahead of systems from labs many times its size.

The competition optimizes for other things. Parallel, a web research company, tunes for long, in-depth answers. Exa builds embedding-based semantic search for discovery rather than verification. Sixtyfour measures something narrower and harder: whether a specific claim about a specific entity is verifiably true.

Then there is the part that compounds. Every investigation Sixtyfour runs resolves the entities it touches into a graph the next investigation can draw on. The more the system is used, the more it already knows.

“The thousandth time you investigate a network of sellers, the graph is doing half the work before an agent even starts, and it lets us track patterns like fraud rings and networks of bad actors all connected together,” Shah said. “That is the part you cannot buy your way to. You earn it one investigation at a time.

The approach is drawing outside notice. In June, Y Combinator invited Shah to speak at its internship event, an in-person talk to hundreds of interns from across its portfolio alongside the firm’s Jared Friedman. “Work on hard problems,” he told them. “As AI tools and agents get better, the alpha shifts to hard problems that require domain expertise and near-perfect accuracy. One wrong call in our domain and someone gets called out, banned, or even arrested.” For a founder whose argument is that reliability has to be built rather than claimed, the invitation to teach it carried its own kind of proof.

Shah wrote much of the original system himself before the company grew to a dozen people, and he talks about correctness the way other founders talk about growth. The bet runs against the grain of a field that rewards fluency: buyers will pay more for a system that knows when it does not know than for one that always has an answer ready. The scoreboard on his screen is how he intends to prove it, one graded run at a time.



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


YouTube has an AI slop problem, and its crackdown is catching legitimate creators in the crossfire. Faceless channels, where no human host ever appears on screen, have existed for years and are not inherently AI-generated.

Many are run by solo creators who simply prefer to stay anonymous. The problem is that AI tools made it easy to flood the platform with low-effort faceless content at scale, and YouTube’s algorithm is now penalizing the format as a whole.

How bad is the AI slop problem on YouTube?

A Kapwing study found that roughly 21% of the first 500 videos recommended to a new YouTube account were classified as AI slop, while 33% fell into a broader brainrot category. The problem extends to children, too, as more than 40% of YouTube Shorts recommended to kids in a 15-minute session contained low-quality AI content.

YouTube’s response has been to tweak its algorithm to favor videos with real human faces on camera, which is hitting faceless creators even when their content is entirely human-made.

How is YouTube tackling its AI slop problem?

YouTube is now testing a new pop-up on mobile that asks viewers to rate whether a video feels like AI slop, on a scale from “not at all” to “extremely.” The idea sounds reasonable, but crowdsourcing AI detection has real problems. People are bad at spotting AI content, and they are getting worse at it as AI capabilities continue to improve.

There are also legitimate concerns that YouTube could use this viewer feedback as training data for its own AI models, potentially making future AI-generated content even harder to spot.

🚨 Did you just see what YouTube did?

YouTube isn’t banning AI slop.. They’re making you label it so they can train their next model to not look like slop.

Read that again…

You flag the bad AI content. YouTube collects it. Google feeds it into Veo 4… Then next year their… https://t.co/8UC2J3mjjv pic.twitter.com/mIrTChqC1b

— Tuki (@TukiFromKL) March 17, 2026

Meanwhile, faceless creators are scrambling to adapt. According to The Hollywood Reporter, some are hiring cheap on-camera hosts through platforms like Fiverr and Upwork. Others are doubling down on niche educational content, which has held up better than broad content farms.

The AI text-to-video space is still valued at enormous sums, with Higgsfield AI alone sitting at $1 billion, but on YouTube, the math for faceless creators is getting harder to work out every month.



Source link