An AWS Machine Learning Blog post demonstrates how to combine Data Version Control (DVC) with Amazon SageMaker AI and Amazon SageMaker AI MLflow Apps to record end-to-end lineage for machine learning artifacts. The post walks through two deployable patterns—dataset-level lineage and record-level lineage—and provides companion notebooks you can run in your AWS account.
- Two deployable lineage patterns: dataset-level and record-level
- Integrates DVC with Amazon SageMaker AI and MLflow Apps for versioned artifact tracking
- Companion notebooks enable hands-on deployment in your AWS account
What happened
AWS published a tutorial demonstrating how to pair DVC (Data Version Control) with Amazon SageMaker AI and Amazon SageMaker AI MLflow Apps to produce end-to-end lineage for ML workflows. The post presents two concrete patterns—one that captures lineage at the dataset level and another that captures it at the record level—and includes deployable examples and companion notebooks designed to run in a user’s AWS account.
Why it matters
Lineage is essential for reproducibility, debugging, and compliance in production ML. Combining DVC’s dataset versioning with SageMaker AI’s tracking and MLflow Apps gives teams a practical route to link datasets, training runs, and model artifacts across the lifecycle. Having deployable patterns for both dataset- and record-level tracing helps teams choose the granularity they need for auditability and root-cause analysis.
What to watch next
Try the companion notebooks in a test AWS account to evaluate which pattern fits your pipeline and governance needs. Watch for updates to the notebooks or additional examples from AWS that extend integrations or cover operationalization at scale. Consider how dataset versus record granularity affects storage, performance, and compliance requirements before adopting a pattern broadly.