Seeking improved read latency and cost efficiency, Perplexity developed CobbleDB, a lightweight Rust key-value store built with AI-assisted engineering. The new database dramatically lowers median and p99 batch read latency on production search queries while reducing cloud expenses by at least 20%.
- Median batch-read latency cut from 31.4 ms to 5.6 ms in production
- Expected cloud cost reduction of at least 20% compared to DynamoDB
- AI agents aided development but humans retain production control
Infrastructure signal
Perplexity’s transition from DynamoDB to CobbleDB reflects a strategic infrastructure shift prioritizing control and performance optimization in cloud service spending. With DynamoDB’s limited configurability on read processing causing latency spikes and costly read/write fees, the in-house database offers finely tuned replication and caching using RocksDB and hashed URL keys. It also spreads replicas across availability zones with fallbacks to avoid slowdowns from single points of lag.
This hybrid architecture decouples long-term document archiving—stored using durable storage solutions like YTsaurus and S3—from the serving layer optimized for low-latency search access. The modular storage pipeline allows independent replica synchronization to reduce blocking and maintain high throughput, reaching up to half a million requests per second in load tests. Such optimizations are critical for managing growing datasets while controlling the explosive costs typical of fully managed cloud NoSQL stores.
Developer impact
Developers leveraged hundreds of AI coding agents to assist with CobbleDB’s creation, accelerating code generation, testing, and problem detection across sessions. However, production deployment control and architectural decisions remained exclusively with the two engineers leading the project. This balance ensured that the AI agents enhanced productivity without risking reliability or data integrity by managing live environments.
The Rust-based design supports embedded memory stores and local NVMe caches to optimize frequent data access patterns, fitting the search workload’s needs. Developers benefit from an open-source promise, contributing to and reusing a proven lightweight, high-performance database tailored for real-time key-value workloads. This approach contrasts with black-box cloud services, offering greater transparency and customizability in deployment and observability workflows.
What teams should watch
Teams facing escalating cloud NoSQL costs and unpredictable performance under managed services should consider hybrid architectures that decouple archival storage from hot serving layers. CobbleDB demonstrates the potential of open-source, purpose-built databases combined with persistent AI-assisted development to drastically improve latency and cost-efficiency in search-heavy applications.
However, organizations must weigh the engineering overhead of building and maintaining custom databases against cloud provider convenience. Keeping human oversight in production is critical to avoid irreversible mistakes inherent with AI-driven automation, especially when handling sensitive production data. Monitoring replication lag, read latencies, and fallback mechanisms is essential for ensuring reliability.
Additionally, watching the ongoing feasibility of expanding AI agent roles beyond code assistance into more autonomous infrastructure management will be important, balancing speed gains with cautious risk management. CobbleDB’s case suggests a hybrid approach—augmenting developer productivity with AI aids while retaining authoritative control—may become a best practice for future cloud infrastructure innovation.