Most AI research tools rely on language models querying the web and trusting the results at face value. Sixtyfour, led by Saarth Shah, takes a different approach: it grades its AI agents rigorously against expert-compiled benchmarks and only ships improvements that prove accuracy, not just fluency.
- Sixtyfour grades AI builds with expert-verified benchmarks before shipping.
- Agents integrate multi-level data sources beyond surface web results.
- Every answer links to source documents ensuring auditability.
What happened
Saarth Shah and his company Sixtyfour have developed an AI research system that rejects the common industry shortcut of simply querying language models on the open web and trusting the generated answers. Instead, every AI build is evaluated against hand-assembled expert questions and real-world cases, with the score determining what software updates get released.
The system integrates a variety of data sources including licensed proprietary records, public filings, sanctions, and litigation databases, and cross-references them in investigations. The output is a comprehensive report where every factual claim cites its source document, designed to meet the evidentiary standards expected in high-stakes scenarios such as payments fraud checks.
Why it matters
In the AI industry, a common problem is the tension between impressive language fluency and factual correctness, especially for intricate domains requiring deep verification. Existing models can hallucinate or produce plausible but incorrect answers, which makes them unreliable for serious business or legal use cases.
Sixtyfour’s approach directly addresses the critical gap between capability and reliability by engineering a system that only delivers verifiably correct results. This is particularly vital because many decisive facts reside beneath the surface of public web pages, in databases and filings unindexed by search engines, limiting typical AI tools’ effectiveness.
What to watch next
SignalDesk and industry watchers should track how Sixtyfour’s evaluation-first methodology influences adoption in sectors that must trust AI decisions—such as payments, compliance, and legal research. Their insistence on linking every answer to source data could set a new standard for accountability in production AI applications.
Further developments will likely involve expanding the benchmarks with more complex real-world cases and broader verticals. As market demand for reliable AI grows, companies that can prove correctness rather than claim it will gain competitive advantage. Monitoring Sixtyfour’s deployment outcomes and updates will reveal how well rigorous evaluation scales in practice.