Diffbrief Engineering daily

← Sunday, October 18, 2026DatabasesDeep dive

BigQuery queries cross-cloud data in place through the Iceberg REST catalog

cloud.google.com by Praveen Kumar 11 min read Editor score 6/10

The described design uses the Apache Iceberg REST catalog to query remote datasets without copying them into a central staging warehouse, avoiding synchronization lag, storage costs and egress fees. It is a useful pattern for cross-cloud analytics and AI workloads.

What
BigQuery can query data in place across Databricks, Snowflake, and AWS Glue using the Apache Iceberg REST catalog specification.
How
Lakehouse runtime catalog periodically syncs remote metadata; BigQuery reads Parquet files directly from remote storage using temporary credentials.
Why it matters
In-place querying avoids extraction pipelines and their synchronization lag, storage costs, egress fees, and schema-change breakage.
metadata metadata metadata catalog Parquet data Databricks UnityCatalog SnowflakeHorizon Catalog AWS Glue DataCatalog Amazon S3 Lakehouseruntime catalog BigQuery
Catalog metadata is synchronized separately from direct reads of remote data files. Sketched from the article's own text.

Read on cloud.google.com →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.

Your note

Private to you. Stored in this browser, and synced to your account when you are signed in.