Bright Machines transforms highly secured, air-gapped factory environments by delivering live, role-based dashboards externally without opening inbound connectivity. By using edge change data capture combined with cloud federation and strict governance, they maintain security and accelerate data freshness.
- Edge CDC and data federation enable secure, near real-time dashboards without inbound access
- Unified, governed access via Unity Catalog ensures fine-grained role-based permissions
- Simplified architecture reduces engineering overhead and supports air-gapped plant constraints
Infrastructure signal
Bright Machines redesigned their data pipeline to support live factory dashboards without violating the air-gapped architecture that forbids inbound network connectivity. They employ Debezium for PostgreSQL change data capture (CDC) on-premises at the factory edge, writing data as Apache Iceberg tables, which are then securely transmitted outbound through a single auditable channel.
The cloud side reads these tables in place through Databricks Unity Catalog federation, enabling governed, fine-grained access that mimics on-premises role-based controls. This approach avoids the need to reconstruct or replicate factory databases in the cloud, reducing operational complexity and cloud compute costs by keeping raw data processing within environments owned by Bright Machines and their customers.
Developer impact
Previously, data engineers spent hours to days manually reshaping data via multiple ETL jobs due to complex pipelines involving several loosely coupled services. The new pipeline streamlines data flow by directly capturing changes edge-side and exposing governed views on the cloud with minimal intervention, thus accelerating development velocity.
Developers now benefit from a simplified, durable data pipeline architecture built on open formats (Apache Iceberg) and well-known CDC tooling (Debezium), which improves maintainability and observability. The use of Unity Catalog for federation provides a consistent security model and reduces complexities around role-based access and auditing, enabling faster iterations of factory performance dashboards.
What teams should watch
Operations, security, and data teams should monitor outbound channel performance and audit logs continuously to verify that no unauthorized data paths emerge, ensuring compliance with air gap constraints. Any deviation in latency or data freshness impacting near real-time dashboard updates should also be promptly addressed to maintain operational insights.
Engineering teams must keep an eye on evolving CDC connectors and metadata federation capabilities, as improvements here could further reduce latencies and extend supported data sources. They should also plan for integration scalability to support additional factories or extended use cases such as quality control, vendor component tracking, and regulatory reporting in an air-gapped context.