> For the complete documentation index, see [llms.txt](https://docs.soda.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.soda.io/guides-and-tutorials/data-consumer/build-a-sigma-dashboard.md).

# Build a Sigma dashboard

Load Soda Cloud check and dataset results into PostgreSQL with the Soda Cloud API, and build a data quality dashboard on them in Sigma.

This guide shows how to build a data quality dashboard in Sigma from your Soda Cloud results. Sigma reads data from a warehouse, not from an API, so a short Python script copies the results of every dataset and check from the Soda Cloud API into two PostgreSQL tables. A Sigma workbook then reports on those tables.

Use the example as a starting point: the same approach works with any warehouse Sigma connects to.

{% hint style="info" %}
If your organization uses the Diagnostics Warehouse, Soda already writes the results into your own warehouse, and Sigma can read them without a script. See [Data quality reporting](broken://spaces/A2PmHkO5cBgeRPdiPPOG/pages/b7114fd9415b44128d58c6f572b99d73bb1bfd36).
{% endhint %}

## Prerequisites

* A Soda Cloud API key. See [Generate API keys](https://docs.soda.io/apis/generate-api-keys.md).
* A PostgreSQL database that Sigma can connect to, and a user that can create a schema and tables in it
* A Sigma account that can create workbooks, and a Sigma connection to the database, or permission to create one

## Load the results into PostgreSQL

The script reads every page of `GET /api/v1/datasets` and `GET /api/v1/checks`, and replaces the content of two tables:

| Table             | One row per | Columns                                                                                                                                                   |
| ----------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `datasets_report` | dataset     | ID, name, qualified name, data source, health status, number of checks, last update, Soda Cloud link                                                      |
| `checks_report`   | check       | ID, name, dataset, column, latest evaluation status (`pass`, `warn`, `fail`, or `notEvaluated`), last run time, check attributes as JSON, Soda Cloud link |

Each row also records when the script extracted it.

{% code title="soda\_to\_postgres.py" %}

```python
"""Copy Soda Cloud dataset and check results into PostgreSQL for Sigma."""
import json
import os
import time
from datetime import datetime, timezone

import psycopg
import requests

SODA = os.environ["SODA_CLOUD_HOST"].rstrip("/") + "/api/v1"
AUTH = (os.environ["SODA_CLOUD_API_KEY_ID"], os.environ["SODA_CLOUD_API_KEY_SECRET"])
SCHEMA = os.environ.get("POSTGRES_SCHEMA", "soda_reporting")


def get_all(path: str) -> list[dict]:
    """Read every page of a Soda Cloud list endpoint."""
    rows, page = [], 0
    while True:
        response = requests.get(f"{SODA}/{path}", auth=AUTH, params={"page": page, "size": 100})
        if response.status_code == 429:
            time.sleep(int(response.headers.get("Retry-After", "10")))
            continue
        response.raise_for_status()
        body = response.json()
        rows.extend(body["content"])
        page += 1
        if page >= body["totalPages"]:
            return rows


def main() -> None:
    extracted_at = datetime.now(timezone.utc)
    datasets = get_all("datasets")
    checks = get_all("checks")

    with psycopg.connect(
        host=os.environ["POSTGRES_HOST"],
        port=os.environ.get("POSTGRES_PORT", "5432"),
        user=os.environ["POSTGRES_USER"],
        password=os.environ["POSTGRES_PASSWORD"],
        dbname=os.environ["POSTGRES_DATABASE"],
    ) as conn, conn.cursor() as cur:
        cur.execute(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
        cur.execute(f"""
            CREATE TABLE IF NOT EXISTS {SCHEMA}.datasets_report (
                dataset_id text PRIMARY KEY, dataset_name text, qualified_name text,
                data_source text, health_status integer, checks integer,
                last_updated timestamptz, cloud_url text, extracted_at timestamptz)""")
        cur.execute(f"""
            CREATE TABLE IF NOT EXISTS {SCHEMA}.checks_report (
                check_id text PRIMARY KEY, check_name text, dataset_id text,
                dataset_name text, column_name text, evaluation_status text,
                last_check_run_time timestamptz, attributes jsonb, cloud_url text,
                extracted_at timestamptz)""")
        cur.execute(f"TRUNCATE {SCHEMA}.datasets_report, {SCHEMA}.checks_report")

        cur.executemany(
            f"INSERT INTO {SCHEMA}.datasets_report VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)",
            [(d["id"], d["name"], d.get("qualifiedName"), (d.get("datasource") or {}).get("name"),
              d.get("healthStatus"), d.get("checks"), d.get("lastUpdated"), d.get("cloudUrl"),
              extracted_at) for d in datasets],
        )
        cur.executemany(
            f"INSERT INTO {SCHEMA}.checks_report VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)",
            [(c["id"], c["name"], (c.get("datasets") or [{}])[0].get("id"),
              (c.get("datasets") or [{}])[0].get("name"), c.get("column"),
              c.get("evaluationStatus"), c.get("lastCheckRunTime"),
              json.dumps(c.get("attributes") or {}), c.get("cloudUrl"), extracted_at)
             for c in checks],
        )
    print(f"Loaded {len(datasets)} datasets and {len(checks)} checks into {SCHEMA}")


if __name__ == "__main__":
    main()
```

{% endcode %}

Install the two packages the script uses, set the environment variables, and run it:

```bash
pip install requests "psycopg[binary]"

export SODA_CLOUD_HOST=https://cloud.us.soda.io
export SODA_CLOUD_API_KEY_ID=<your_api_key_id>
export SODA_CLOUD_API_KEY_SECRET=<your_api_key_secret>
export POSTGRES_HOST=<your_host>
export POSTGRES_USER=<your_user>
export POSTGRES_PASSWORD=<your_password>
export POSTGRES_DATABASE=<your_database>
export POSTGRES_SCHEMA=soda_reporting

python soda_to_postgres.py
```

Use `https://cloud.soda.io` for `SODA_CLOUD_HOST` if your organization is in the EU region. The script prints how many datasets and checks it loaded.

To keep the dashboard current, run the script on a schedule, for example in the same orchestrator that runs your other data jobs.

> For every field the endpoints return, see the [Soda Cloud REST API](https://docs.soda.io/apis/rest-api.md).

## Connect Sigma to the database

If Sigma doesn't have a connection to the database yet, an admin creates one. See [Connect to PostgreSQL](https://help.sigmacomputing.com/docs/connect-to-postgresql) in the Sigma documentation.

If the connection already exists, open it from **Connections** and click **Sync connection metadata**, so that Sigma lists the new schema and tables.

## Build the workbook

Create a workbook and add the two tables from the connection as data elements. See [Create a workbook](https://help.sigmacomputing.com/docs/create-a-workbook) in the Sigma documentation.

On top of the tables, build the elements of your dashboard. For example:

* A KPI chart with the share of checks that pass, with the formula `CountIf([Evaluation Status] = "pass") / Count()` formatted as a percentage. See [Build a KPI chart](https://help.sigmacomputing.com/docs/build-a-kpi-chart).
* A bar chart with the number of checks per dataset, colored by evaluation status. See [Build a bar chart](https://help.sigmacomputing.com/docs/build-a-bar-chart).
* A table of failing checks, with the Soda Cloud link of each check, so a reader can open the check's history in one click.

<figure><img src="https://1928210810-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJJOow5YZtrrxZJBNXV2D%2Fuploads%2Fgit-blob-2fbb0ca1ba80e898bd9385c4ef5aff088b146020%2Fbuild-a-sigma-dashboard-sigma-workbook.png?alt=media" alt="A Sigma workbook with a KPI with the share of passing checks, a bar chart of checks per dataset by evaluation status, and a table of failing checks with their Soda Cloud links"><figcaption><p>A data quality workbook built on the two tables</p></figcaption></figure>

## Next steps

* Use check attributes to group the dashboard by team or domain. See [Check and dataset attributes](https://docs.soda.io/soda-documentation/govern/check-and-dataset-attributes.md).
* Read the full API reference for more fields to report on. See [Soda Cloud REST API](https://docs.soda.io/apis/rest-api.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.soda.io/guides-and-tutorials/data-consumer/build-a-sigma-dashboard.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
