Al is increasing the complexity of modern data platforms, but many organizations are building systems with limited visibility into how data moves, changes, and is consumed.
This talk explores approaches for building observable and portable data platforms using open technologies and open standards. Through real-world examples from large-scale enterprise environments, we will look at how tools and architectures based on Delta Lake, Apache Spark, and OpenLineage can improve lineage tracking, governance, interoperability, and operational transparency across modern data pipelines.
The session covers common problems in Al-oriented data architectures, including fragmented metadata, hidden operational costs, and observability. We will also discuss practical engineering patterns for building lakehouse platforms that remain flexible, traceable, and easier to operate over time.
