Flo Health, a leader in women's health with 81 million monthly users, transitioned to fine-tuned open generative AI models using Databricks AI Runtime. This approach enhances medical safety and personalization while dramatically lowering cloud inference costs.
- Fine-tuned models cut inference costs roughly tenfold versus major API usage
- Medical safety rigor encoded via AI judges and synthetic training data
- Databricks AI Runtime leverages serverless H100 GPUs for scalable training
Infrastructure signal
Flo Health deployed Databricks AI Runtime to enable distributed fine-tuning on serverless NVIDIA H100 GPUs, optimizing cost-efficiency and scalability. This cloud infrastructure supports training with synthetic datasets derived under privacy-first principles, allowing for large-scale customization without compromising user privacy.
By migrating away from pay-as-you-go API models to in-house fine-tuning pipelines, Flo Health reduced inference expenses by an order of magnitude. The combination of scalable GPU resources and synthetic data generation improves operational reliability and decreases dependency on external model providers' update cycles.
Developer impact
Flo’s engineering and medical teams collaborated to define medical safety and usefulness standards, embedding domain expertise into AI judges that evaluate model responses automatically. This evaluation framework replaced manual human labeling, accelerating the training data pipeline through synthetic question generation and reward-driven fine-tuning.
Developers now leverage a reinforcement fine-tuning approach that iteratively ranks candidate responses using AI judges as reward signals. This workflow not only improves model alignment with Flo’s safety criteria but also streamlines iterative model improvements, reducing clinical experts’ manual labor and shortening training cycles.
What teams should watch
Teams should monitor evolving AI evaluation rubrics and judge performance to maintain safety and usefulness as the product scales. Ongoing calibration and validation using synthetic and anonymized user data are essential to keep models aligned with medical standards and regulatory compliance.
Product and data science teams must coordinate closely with infrastructure to optimize GPU usage patterns, synthetic data generation, and deployment pipelines, ensuring cost-efficiency and reliable observability metrics. Tracking inference latency and accuracy across model variants will guide platform decisions on future fine-tuning and API usage levels.