Diffbrief Engineering daily

← Tuesday, October 20, 2026DatabasesGuide

BigQuery Python UDFs can send structured logs to Cloud Logging

cloud.google.com by Cong Du, Ming Ma 5 min read Editor score 7/10

Python UDFs can use the standard logging library to report row-level errors to Cloud Logging, allowing bad records to be isolated without halting a query. The end-to-end example shows a practical observability pattern for resilient data pipelines.

What
BigQuery Python UDFs can use Python’s standard logging library to send structured logs to Cloud Logging without external plugins or connectors.
How
The example validates JSON rows, routes valid records to a destination table, and logs bad rows for alerting and SQL triage.
Why it matters
Row-level diagnostics let teams isolate problematic records and investigate pipeline failures without halting workloads.
valid rows errors log metric linked logs SQL triage Python UDF destinationtable Cloud Logging Cloud Monitoring _AllLogs view BigQuery
The UDF routes valid rows to a table and logs errors for alerting and SQL triage. Sketched from the article's own text.

Read on cloud.google.com →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.

Your note

Private to you. Stored in this browser, and synced to your account when you are signed in.