Trillium Labs, founded by AI scientists Nathan Lambert and Tom Zick, is launching with a mission to openly share the details of complex AI research often kept secret by leading industry labs. Their goal is to foster community scrutiny and safer AI development through public experimentation and model analysis.
- Trillium Labs promotes open publication of risky AI experiments.
- Focus areas include RSI, post-training, and reinforcement learning.
- Transparency aims to enhance community involvement and safety.
What happened
Nathan Lambert and Tom Zick have launched Trillium Labs, a nonprofit dedicated to conducting and sharing research on advanced AI topics that are often shrouded in secrecy. By publishing detailed experimental data and methods, they aim to enable external scientists to replicate and inspect complex AI model development.
Their research focus includes areas considered risky or contentious, such as recursive self-improvement—the iterative self-upgrading of AI models—and the use of reinforcement learning to influence AI behavior. Trillium Labs positions itself against the current industry's closed practices, seeking an open scientific method ethos to improve transparency and safety.
Why it matters
Most leading AI models, from companies like OpenAI and Anthropic, remain accessible only through controlled APIs, limiting researchers’ ability to study underlying architectures and training processes. This secrecy challenges academic verification and risk assessment, especially as AI systems gain capabilities with potentially high stakes.
By openly sharing research, Trillium Labs intends to democratize insights on AI safety and behavior, enabling a broader community to monitor, critique, and innovate on AI development. This could improve our ability to foresee and mitigate unintended consequences, particularly in areas like RSI, which some experts warn could threaten human oversight.
What to watch next
Trillium Labs plans to concentrate initially on post-training improvements and detailed studies of reinforcement learning's effects on AI agents' capabilities and decision-making. Their experiments will aim to provide new understanding of how these techniques shape model behaviors, including potentially problematic traits like excessive sycophancy.
As the nonprofit establishes itself, the AI field will observe whether its open model leads to new collaborative breakthroughs and safety protocols. The wider debate—between limited-access control versus transparent research—will likely intensify as Trillium Labs’ approach challenges the status quo of frontier AI development.