> For the complete documentation index, see [llms.txt](https://docs.soda.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.soda.io/guides-and-tutorials/data-owner-producer/test-data-during-development.md).

# Test data during development

Verify data contracts on every pull request with GitHub Actions, so a change that breaks the data is caught before it merges.

This guide shows how to verify Soda data contracts in a GitHub Actions workflow that runs on every pull request. The workflow builds the dbt models in a CI schema, verifies their contracts, and posts the results to the pull request. When a check fails, the job fails, so the change is caught before it reaches production.

Use the example as a starting point and adapt the data source, models, and contracts to your own repository.

## Example repository

A team keeps its dbt models and their Soda contracts in the same repository. One model, `fact_product_category`, joins products to their subcategories and categories. Its contract requires every product to appear exactly once.

A pull request that changes the model triggers the workflow, which:

1. Builds the models with dbt in a CI schema, separate from production
2. Verifies every contract against the CI schema
3. Writes the results to the job summary and posts them as a comment on the pull request
4. Fails the job if any check fails

The repository has this layout:

```
.github/
└── workflows/
    └── verify-data-contracts.yml
dbt/
├── dbt_project.yml
├── profiles.yml
└── models/
    ├── sources.yml
    └── fact_product_category.sql
soda/
├── ds_config.yml
└── contracts/
    └── fact_product_category.yml
```

## Prerequisites

* A GitHub repository with GitHub Actions enabled
* A PostgreSQL data source that GitHub-hosted runners can reach, with a schema the workflow can write to for CI builds
* A dbt project that builds models in that data source

## Configure the data source

Create a data source configuration file in the `soda` folder. The values in `${env.…}` are read from environment variables, which the workflow sets from the repository's secrets and variables.

{% code title="soda/ds\_config.yml" %}

```yaml
type: postgres
name: warehouse
connection:
  host: ${env.POSTGRES_HOST}
  port: 5432
  user: ${env.POSTGRES_USER}
  password: ${env.POSTGRES_PASSWORD}
  database: ${env.POSTGRES_DATABASE}
```

{% endcode %}

dbt reads the same environment variables. The CI target builds models in the schema named by `CI_SCHEMA`, and the sources read from `SOURCE_SCHEMA`:

{% code title="dbt/profiles.yml" %}

```yaml
sales:
  target: ci
  outputs:
    ci:
      type: postgres
      host: "{{ env_var('POSTGRES_HOST') }}"
      port: 5432
      user: "{{ env_var('POSTGRES_USER') }}"
      password: "{{ env_var('POSTGRES_PASSWORD') }}"
      dbname: "{{ env_var('POSTGRES_DATABASE') }}"
      schema: "{{ env_var('CI_SCHEMA') }}"
      threads: 1
```

{% endcode %}

{% code title="dbt/models/sources.yml" %}

```yaml
version: 2
sources:
  - name: adventureworks
    schema: "{{ env_var('SOURCE_SCHEMA') }}"
    tables:
      - name: dim_product
      - name: dim_product_category
      - name: dim_product_subcategory
      - name: fact_internet_sales
```

{% endcode %}

> For the configuration keys of each data source, see the [Data source reference for Soda Core](https://docs.soda.io/soda-documentation/developer-tools/data-source-reference-for-soda-core.md).

## Write the contract

The contract uses two variables, `DATABASE` and `SCHEMA`, in the dataset name. The workflow sets them to the CI schema, so the same contract file can verify the production table when you pass the production schema instead.

{% code title="soda/contracts/fact\_product\_category.yml" %}

```yaml
dataset: warehouse/${var.DATABASE}/${var.SCHEMA}/fact_product_category

variables:
  DATABASE:
  SCHEMA:

columns:
  - name: product_key
    checks:
      - missing:
          name: All products have a key
      - duplicate:
          name: Each product appears once
  - name: english_product_name
  - name: subcategory_key
  - name: category_key
    checks:
      - missing:
          name: All products have a category
          threshold:
            must_be: 0
            level: warn
  - name: category_name

checks:
  - schema:
```

{% endcode %}

A product without a category isn't an error in this model, so that check warns instead of failing.

> To learn more about variables, see [Make contracts dynamic with variables](https://docs.soda.io/soda-documentation/developer-tools/contract-language-reference.md#make-contracts-dynamic-with-variables).

## Create the workflow

Add the workflow file to `.github/workflows`:

{% code title=".github/workflows/verify-data-contracts.yml" %}

````yaml
name: Verify data contracts

on:
  pull_request:

permissions:
  contents: read
  pull-requests: write

concurrency:
  group: data-contracts-ci

jobs:
  verify:
    runs-on: ubuntu-latest
    env:
      POSTGRES_HOST: ${{ secrets.POSTGRES_HOST }}
      POSTGRES_USER: ${{ secrets.POSTGRES_USER }}
      POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
      POSTGRES_DATABASE: ${{ vars.POSTGRES_DATABASE }}
      SOURCE_SCHEMA: ${{ vars.SOURCE_SCHEMA }}
      CI_SCHEMA: ${{ vars.CI_SCHEMA }}
    steps:
      - uses: actions/checkout@v7

      - uses: actions/setup-python@v7
        with:
          python-version: "3.12"

      - name: Install dbt and Soda
        run: |
          pip install dbt-core dbt-postgres
          pip install -i https://pypi.cloud.soda.io/simple "soda-postgres>4"

      - name: Build the models in the CI schema
        run: dbt build --project-dir dbt --profiles-dir dbt

      - name: Verify the contracts
        id: verify
        run: |
          set +e
          failed=false
          {
            echo "## Soda contract verification"
            for contract in soda/contracts/*.yml; do
              output=$(soda contract verify -ds soda/ds_config.yml -c "$contract" \
                --set DATABASE="$POSTGRES_DATABASE" --set SCHEMA="$CI_SCHEMA" 2>&1)
              code=$?
              case $code in
                0) status="passed" ;;
                2) status="passed with warnings" ;;
                1) status="failed"; failed=true ;;
                *) status="error (exit code $code)"; failed=true ;;
              esac
              echo "### \`$contract\`: $status"
              echo '```text'
              echo "$output" | sed -n '/^### Contract results/,$p' | grep . || echo "$output" | tail -20
              echo '```'
            done
          } > results.md
          cat results.md >> "$GITHUB_STEP_SUMMARY"
          echo "failed=$failed" >> "$GITHUB_OUTPUT"

      - name: Comment on the pull request
        env:
          GH_TOKEN: ${{ github.token }}
        run: gh pr comment ${{ github.event.pull_request.number }} --body-file results.md

      - name: Fail the job if a contract failed
        if: steps.verify.outputs.failed == 'true'
        run: exit 1
````

{% endcode %}

A few details in the workflow:

* `soda contract verify` exits with `2` when checks only warn. The verify step treats `2` as a pass, and `1` or any other code as a failure. See [Exit codes](https://docs.soda.io/soda-documentation/developer-tools/cli-reference.md#exit-codes) in the CLI reference.
* The verify step runs every contract before it reports, so one failing contract doesn't hide the results of the others. The last step fails the job.
* `pull-requests: write` lets the workflow post the comment with the built-in `GITHUB_TOKEN`.
* `concurrency` runs one workflow at a time, because every pull request builds in the same CI schema.

{% hint style="warning" %}
GitHub doesn't pass secrets to workflows that run on pull requests from forks, so on those pull requests the workflow can't connect to the data source. See [Using secrets in GitHub Actions](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions) in the GitHub documentation.
{% endhint %}

## Add the secrets and variables

In the repository, open **Settings** > **Secrets and variables** > **Actions**, and add the values the workflow reads:

| Name                | Kind     | Value                                              |
| ------------------- | -------- | -------------------------------------------------- |
| `POSTGRES_HOST`     | Secret   | Host name of the PostgreSQL server                 |
| `POSTGRES_USER`     | Secret   | User that builds the CI schema and runs the checks |
| `POSTGRES_PASSWORD` | Secret   | Password of that user                              |
| `POSTGRES_DATABASE` | Variable | Database name                                      |
| `SOURCE_SCHEMA`     | Variable | Schema of the source tables                        |
| `CI_SCHEMA`         | Variable | Schema where the workflow builds the models        |

## Open a pull request

Open a pull request that changes a model. The workflow runs, and its results appear in three places: the check on the pull request, a comment on the pull request, and the job summary.

When a check fails, the job fails. For example, a change that joins `fact_internet_sales` into `fact_product_category` without aggregating it repeats each product once per sale. dbt builds the model without an error, but the contract's **Each product appears once** check fails, and the pull request shows a failed check.

<figure><img src="https://1928210810-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJJOow5YZtrrxZJBNXV2D%2Fuploads%2Fgit-blob-fc8d0104386f9c8c5afed1c1a4fbf7fd7b6bdcb0%2Ftest-data-during-development-github-pr-failed-check.png?alt=media" alt="The checks box of a pull request in GitHub, with the Verify data contracts check failed"><figcaption><p>The failed check on the pull request</p></figcaption></figure>

<figure><img src="https://1928210810-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FJJOow5YZtrrxZJBNXV2D%2Fuploads%2Fgit-blob-37078a066f7f9a1fdb3a34b0bcaf9e48776ba315%2Ftest-data-during-development-github-pr-comment.png?alt=media" alt="The comment the workflow posts on the pull request, with the contract results table and the failed check Each product appears once"><figcaption><p>The comment the workflow posts, with the contract results</p></figcaption></figure>

To block merging while the check fails, make the `verify` job a required status check in a branch protection rule. See [About protected branches](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) in the GitHub documentation.

## Next steps

* Publish contracts to Soda Cloud when they merge, with the [GitHub integration](https://docs.soda.io/soda-documentation/integrate/github.md).
* Verify the same contracts in your production pipeline. See [Test data quality in an Airflow pipeline](https://app.gitbook.com/s/JJOow5YZtrrxZJBNXV2D/data-owner/test-data-quality-in-an-airflow-pipeline).
* Learn every check type in the [Contract Language reference](https://docs.soda.io/soda-documentation/developer-tools/contract-language-reference.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.soda.io/guides-and-tutorials/data-owner-producer/test-data-during-development.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
