Data Federation using Apache Trino and an Application for Environmental Data

Today, data is increasingly distributed across independent systems managed by different providers, each exposing its data through different technologies and interfaces. Integrating these heterogeneous data sources into a unified view has become essential for data retrieval, analysis, and decision-making. However, achieving this integration remains challenging due to differences in data models (e.g., relational tables and JSON documents), access mechanisms (e.g., REST APIs, web services, or downloadable datasets), metadata structures (e.g., different naming conventions and attribute definitions), and query interfaces. Traditional integration approaches often rely on data replication, leading to increased storage requirements and synchronization overhead.
Federated database systems address this challenge by providing a unified query interface over distributed data sources while allowing each source to remain independent. Instead of replicating data into a central repository, queries are executed across heterogeneous data sources, and the results are combined transparently. Apache Trino is an open-source distributed query engine that enables this approach across many different storage technologies.
This talk presents Apache Trino and research efforts on building a provider-agnostic data federation architecture for heterogeneous environmental data using Apache Trino. The proposed architecture allows access to distributed data sources through a unified global schema without replicating the underlying data. Rather than requiring data providers to migrate or duplicate their datasets, the architecture allows them to expose their existing data sources while preserving source autonomy.
Apache Trino is an open-source distributed SQL query engine designed for federated querying across heterogeneous data sources. The talk provides an overview of Trino’s federation architecture and query execution model, and demonstrates how these capabilities can be used as the foundation of a provider-agnostic architecture for integrating heterogeneous environmental data.
Moreover, the talk demonstrates a provider-agnostic federation architecture built on Apache Trino, consisting of an adapter layer and a unified global schema. The adapter layer enables the integration of heterogeneous data providers by handling provider-specific communication, schema mapping, and query translation, while preserving source autonomy and minimizing data replication. The design of the global schema is supported by a prior analysis of publicly available environmental datasets, resulting in a standardized metadata model that supports the integration of heterogeneous environmental data sources.
The presentation will discuss Apache Trino, the proposed architecture, key design decisions, implementation challenges, and the solutions adopted to build a scalable and extensible federated data architecture for environmental data.