Diagnostics Warehouse

Diagnostics Warehouse provides a clear, detailed view of the state of data checks while allowing access to failed rows in order to take a closer look and resolve data quality issues.

Overview

Diagnostics Warehouse stores all Soda scan details, failed records, and historical data quality issues directly in your data warehouse of choice, safely and securely. Nothing is stored outside. This gives you the ability to run diagnostics, resolve issues, and see exactly why problems happen. You can go as deep as you need, from a single record to a full dataset.

Each time a Soda scan runs, Diagnostics Warehouse stores failed rows together with check and scan results, and related metadata attributes. With that information, data teams can quickly diagnose and resolve issues at both dataset and row level. Additionally, Soda's Diagnostics Warehouse makes it easier for teams to build on top of Soda's outputs to set up operational workflows, and connect to BI tools you already know and trust.

Features & capabilities

• Full diagnostic information in one place, including attributes.

• Transparency for all: replace black-box runs with auditable facts and keep an immutable, queryable history of what was checked, when, how long it took, what failed, and why.

• Faster root-cause analysis: jump from a failed check to the exact failed rows, affected datasets/columns, and prior history to see if it’s a one-off issue or a pattern.

• Operational excellence: monitor failure rates, flaky checks, and run performance. Set SLOs for data quality, and measure MTTR and improvement over time.

• Organization-level visibility: roll up results by domain, team, or pipeline. Show the impact of your data quality program to leadership with real, defensible metrics.

• Open & portable features: it’s just tables in your warehouse. Query with SQL, power dashboards, join with lineage, incident, or cost data, and automate workflows.

• Security & Governance: Diagnostics Warehouse stores tables in your own warehouse, giving you full control over security, retention and access.

Security & governance

• Data minimization: Diagnostics Warehouse stores metadata about runs and checks and, for row-level checks, it only stores failed rows when the option is enabled.

• Warehouse residency: Diagnostics are not stored in Soda. They live in your analytics warehouse, respecting your access controls, encryption, and audit trails.

---

Get started

1. Enable Diagnostics Warehouse in your Soda database settings.

2. Grant the service identity permission to create and write to the Diagnostics Warehouse schema in your warehouse.

3. Run your checks; Diagnostics Warehouse tables populate automatically.

4. Query your warehouse and connect to your BI tools to start exploring.

Next: to enable Diagnostics Warehouse in your organization, reach out to Soda at [email protected].