OpenAI recently published hundreds of AI-generated solutions to challenging mathematical problems but has faced criticism from leading mathematicians for not fully meeting established standards that ensure human understanding and rigorous validation.
- Only 10 of 719 solutions included detailed AI reasoning chains.
- Less than half of proofs underwent formal verification by humans.
- Experts call for halting proprietary model testing on advanced math problems.
What happened
OpenAI released a large batch of AI-generated solutions to some of the most difficult open problems in mathematics. These efforts followed consultations with the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), a consortium of nine prominent mathematicians affiliated with leading institutions, formed to guide frontier labs on responsible approaches in this field.
Despite some adherence to AGMAI’s recommendations—such as prompt sharing of results and some level of explanation—OpenAI fell short on key standards, including insufficient formal verification and lack of comprehensive insight into how models derived their solutions. Only a small fraction of proofs included detailed chain-of-thought disclosures, and fewer than half were formally verified, prompting concerns from the community.
Why it matters
Mathematics relies heavily on deep human understanding and rigorous validation to ensure proofs are correct and meaningful to other researchers. AGMAI's guidelines stress that AI-generated proofs must be transparent and accessible to human mathematicians who validate and integrate these findings into the broader body of knowledge.
A recent study by Cambridge and King’s College mathematicians highlighted discrepancies between AI-produced natural language proofs and the corresponding formalized code, illustrating risks in over-relying on AI’s automatic formalization without thorough human checks. This raises important questions about the readiness of current AI methods to autonomously solve complex mathematical problems to the satisfaction of the academic community.
What to watch next
The advisory group and mathematicians urge OpenAI to cease testing advanced mathematical problems on proprietary black-box models until clearer standards and transparency improve. They also call for the inclusion of machine-readable metadata linking narrative explanations to formal proofs, which OpenAI has yet to implement widely.
Going forward, the development of collaborations that fund and empower human mathematicians to interpret and verify AI-generated proofs will be critical. The field will closely watch how OpenAI and other labs respond to these concerns and whether future releases enhance clarity, reproducibility, and community engagement.