> For the complete documentation index, see [llms.txt](https://docs.soda.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.soda.io/quickstart.md).

# Quickstart

Learn how to get started with Soda.

This quickstart shows how Soda **detects unexpected data issues** by leveraging AI powered Anomaly Detection and **prevents future problems** by using data contracts.

### Scenario

A data engineer at a retail company needs to maintain the `regional_sales` dataset so their team can manage regional sales data from hundreds of stores across the country. The dataset feeds executive dashboards and downstream ML models for inventory planning. Accuracy and freshness are critical, so you need both:

* **Automated anomaly detection** on key metrics (row counts, freshness, schema drift)
* **Proactive enforcement** of business rules via data contracts

### Sign up

> Contact us at <support@soda.io> to get an account set up.

After signing up, you can follow the steps below to set up a data source and start improving data quality.

### Add a Data Source

Soda AI, paired with Soda Cloud’s no-code UI, lets you **connect** to any data source, **onboard** datasets, and **monitor** results in minutes.

{% stepper %}
{% step %}

#### Establish connection

Navigate to the **Data Sources** > Create a **New data source**

**F**ill out the connection details. Then, click on **Connect**.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FbcAI6jiza4zQzDmpIXH6%2Fimage.png?alt=media&amp;token=922cbda8-31a2-4d06-b97a-71041d74152d" alt="" width="375"><figcaption></figcaption></figure>

> Learn more details on how to [Onboard data sources & datasets](/onboard-data-sources-and-datasets.md).
> {% endstep %}

{% step %}

#### Onboard datasets

You can onboard datasets by asking **Soda AI** to do it for you.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FT73M6GqOzG06SIzafIM5%2Fimage.png?alt=media&amp;token=9c645348-0f94-4d7c-baac-11b2fa319af8" alt="" width="298"><figcaption></figcaption></figure>

> If you prefer to onboard datasets manually, head to the [onboarding page](/onboard-data-sources-and-datasets/onboard-datasets-on-soda-cloud.md#onboard-datasets).
> {% endstep %}

{% step %}

#### Enable Metric Monitoring

You can enable Metric Monitoring in several ways:

* While [manually onboarding](/onboard-data-sources-and-datasets/onboard-datasets-on-soda-cloud.md)
* By navigating to the dataset > **Metric Monitoring** tab
* By asking Soda AI to enable Metric Monitoring

{% hint style="info" %}
If you onboarded the dataset via Soda AI, the **dataset page** won't show any monitors yet.
{% endhint %}

Soda AI can **enable Metric Monitoring** for you:

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FVf4gydkxCvu7n7of7fGG%2Fimage.png?alt=media&amp;token=ad2b74d5-ecce-4ec5-8066-9fb3743557f2" alt="" width="447"><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

***

### Review Anomaly Detection results

Congratulations, you’ve onboarded your first dataset! Now let’s make sure you always know what’s happening with it.

That’s where Metric Monitoring comes in. It automatically tracks key metrics like volume, freshness, and schema changes, with no manual setup required. You’ll spot anomalies, detect trends, and catch unexpected shifts before they become problems.

{% stepper %}
{% step %}

#### Open the Metric Monitors dashboard

{% stepper %}
{% step %}
Go to **Datasets** → select the dataset to inspect.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FBFltKODftxkbN76RdVAy%2Fimage.png?alt=media&amp;token=5fe952d1-ec4d-4b50-a12f-61f7ac46a135" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
Navigate to the **Metric Monitors** tab to learn more about the metrics calculated.

If enabled, key metrics are automatically monitored by default, helping you detect pipeline issues, data delays, and unexpected structural changes as they happen.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FrHXt2xyop3tQUa6RAFOE%2Fimage.png?alt=media&amp;token=2c94f46e-4b95-48a6-a4a5-544f0f07ffe3" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}
{% endstep %}

{% step %}

#### View anomalies in a specific monitor

In this guide, we will focus on the **Most recent timestamp** monitor. The panel shows that it was expected to be in a range of **0 - 5m 31s**, but the recorded value at scan time was **56m 49s**. In order to take a closer look:

{% stepper %}
{% step %}
Click the **Most recent timestamp** (or monitor of your choice) block.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FGrOIIMnoj1DrJXLsdGNl%2Fimage.png?alt=media&amp;token=92d0e9c5-1d83-44e7-af08-c82eccd9b99d" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
In the monitor page you’ll see:

* any red-dot anomalies flagged by the model,
* measured value vs. expected range,
* buttons to **Mark as expected**, **Create new incident**, etc.
  {% endstep %}

{% step %}
Flag an outlier as "expected" or investigate it further.
{% endstep %}
{% endstepper %}
{% endstep %}
{% endstepper %}

**Soda’s anomaly detection engine was built in-house** (no third-party libraries) and optimized for high precision. It continuously adapts to your data patterns, and it incorporates your feedback to reduce false alarms. Designed to minimize false positives and missed detections, it shows a 70% improvement in detecting anomalous data quality metrics compared to Facebook Prophet across hundreds of diverse, internally curated datasets containing known data quality issues.

The Anomaly Detection Algorithm offers complete control and transparency in the modeling process to allow for interpretability and adaptations. It features high accuracy while leveraging historical data, delivering improvements over time.

***

### Attack the issues in Soda Cloud

Our **automated anomaly detection** has just done the heavy lifting for you, identifying unusual patterns and potential data issues without any setup required.

But to **prevent** those issues from happening again, you must define exactly what your data should look like; every column, every rule, every expectation.

That’s where **Data Contracts** come in. They let you proactively set the standards for your data, so problems like this are flagged or even prevented before they impact your business.

{% stepper %}
{% step %}

#### Create a Data Contract

Create a new data contract to define and enforce data quality expectations.

{% stepper %}
{% step %}
In your [**dataset page**](/manage-issues/dataset-page.md), go to the **Checks** tab.
{% endstep %}

{% step %}
Click **Create Contract**.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FpI4YqRAjmoaaRrn0qfZn%2Fimage.png?alt=media&amp;token=0c3cc0f6-6ae7-4e5d-b496-3cb4c54fe03e" alt=""><figcaption></figcaption></figure>

When creating a data contract, [Contract Autopilot](/soda-ai/contract-autopilot.md) will connect to your dataset and build a data contract.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FknS54808uGh0lV4B23kD%2Fimage.png?alt=media&amp;token=3159fe07-928d-4ed4-a660-01076689577e" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
\[Optional] Ask [**Contract Copilot**](/soda-ai/contract-copilot.md) to edit the contract.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2Fx3ztVRmCuVPQwoM5W7GA%2Fimage.png?alt=media&amp;token=0b056d52-9bae-4eb0-a741-436fe82e9401" alt=""><figcaption></figcaption></figure>

With Soda, you can edit a contract:

* via [Contract Copilot](/soda-ai/contract-copilot.md),
* using a **no-code interface**, or
* **directly in code**.

This ensures an optimal experience for all users while also providing a **version-controlled code format** that can be synced with a Git repository.
{% endstep %}
{% endstepper %}
{% endstep %}

{% step %}

#### Publish & verify

{% stepper %}
{% step %}
Click **Test** to verify the contract executes as expected
{% endstep %}

{% step %}
When you are done with the contract, click **Publish**

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FzRM5UwkO13qY5eIEyAu0%2Fimage.png?alt=media&amp;token=9df3df66-055c-4c2f-a157-0e07f22fb833" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
Click **Verify all checks**.

Soda will evaluate your rules against the current data.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FgU5RzT9XcQiQhBmCpKkb%2Fimage.png?alt=media&amp;token=f026c56f-c198-46c9-a66f-dd2cff441129" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}
{% endstep %}

{% step %}

#### Review check results

Review the outcomes of the contract checks to confirm whether the data meets expectations.

You can drill into those failures in the **Checks** tab.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FXi1JRkTJvhns8VviwXo7%2Fimage.png?alt=media&amp;token=70e92950-9ec7-4328-9b06-c2cfa7da299b" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

***

### Attack the issues in your pipeline

The same contract you published in Soda Cloud can be verified from your **terminal** or from a **pipeline step**. Run it after a transformation and before anything downstream reads the table, and bad data stops where it was produced.

Three tools do this. Pick by where the verification runs.

| Tool                     | Use it when                                                                                                      |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------- |
| **Soda CLI** (`sodacli`) | You want one tool for the pipeline and the platform. A single Go binary, no dependencies, structured exit codes. |
| **Core CLI** (`soda`)    | You want local execution in a Python environment, with fine-grained control over which checks run.               |
| **Python API**           | Verification is one step inside Python code you already maintain, such as an Airflow task or a notebook.         |

{% tabs %}
{% tab title="Soda CLI" %}
{% stepper %}
{% step %}
**Install**

```bash
brew tap sodadata/tap
brew install sodacli
```

Confirm it with `sodacli version`.

> Windows and install-script options are on the [Soda CLI reference](/reference/soda-cli-reference.md).
> {% endstep %}

{% step %}
**Authenticate**

```bash
sodacli auth login
sodacli auth status
```

`auth login` prompts for host, API key ID and secret, and stores them in `~/.soda/credentials`. In CI, pass `--api-key-id`, `--api-key-secret` and `--no-interactive` instead.

> Generate the keys from your profile: [Generate API keys](/reference/soda-apis/generate-api-keys.md).
> {% endstep %}

{% step %}
**Verify the contract**

```bash
sodacli contract verify databricks_demo/unity_catalog/demo_sales_operations/regional_sales
```

This runs the checks on a Soda Runner. To run them locally through Soda Core instead, add `--local --datasource datasource.yml`.
{% endstep %}

{% step %}
**Review the results**

```bash
sodacli results list --status failing
sodacli job logs <scan-id>
```

Output is a table in a terminal and JSON when piped. Force either with `--output json` or `--output csv`.
{% endstep %}
{% endstepper %}
{% endtab %}

{% tab title="Core CLI" %}
{% stepper %}
{% step %}
**Install**

Install the `soda` package and the connector for your data source, for example `soda-databricks` or `soda-postgres`.

{% hint style="warning" %}
The `soda` package ships from a private PyPI index and needs a Team or Enterprise license, unless you run verifications on a Soda Runner. Contact <support@soda.io> for access.
{% endhint %}

> Installation flow and package list: [Soda Python Libraries](/deployment-options/soda-python-libraries.md).
> {% endstep %}

{% step %}
**Connect to Soda Cloud**

```bash
soda cloud create -f sc_config.yml
```

Fill in the generated file:

{% code title="sc\_config.yml" %}

```yaml
soda_cloud:
  host: cloud.soda.io          # Use cloud.us.soda.io for the US region
  api_key_id: ${env.SODA_API_KEY_ID}
  api_key_secret: ${env.SODA_API_KEY_SECRET}
```

{% endcode %}

Test it with `soda cloud test -sc sc_config.yml`.
{% endstep %}

{% step %}
**Verify the contract**

Fetch the contract you published in Soda Cloud, then verify it:

{% code overflow="wrap" %}

```bash
soda contract fetch -d databricks_demo/unity_catalog/demo_sales_operations/regional_sales -f contract.yaml -sc sc_config.yml

soda contract verify -ds ds_config.yml -c contract.yaml --publish
```

{% endcode %}

`--publish` sends the results back to Soda Cloud, so a pipeline run shows up next to the scheduled ones. It requires the **Manage contract** permission on the dataset.
{% endstep %}

{% step %}
**Run a subset of checks**

Verifying the whole contract on every pipeline run is often more than you need. Select checks by type, column or attribute:

```bash
soda contract verify -ds ds_config.yml -c contract.yaml -cf attributes.severity=critical
```

> Filters, wildcards and check paths: [Core CLI reference](/reference/cli-reference.md)
> {% endstep %}
> {% endstepper %}
> {% endtab %}

{% tab title="Python API" %}
{% stepper %}
{% step %}
**Install and configure**

Install the `soda` package, then create `sc_config.yml` as shown in the Core CLI tab. The Python API reads the same file. {% endstep %}
{% endstep %}

{% step %}
**Verify the contract**

Create `verify_contract.py`:

```python
from soda_core import configure_logging
from soda_core.contracts import verify_contracts_on_runner

configure_logging(verbose=False)

result = verify_contracts_on_runner(
    dataset_identifiers=["databricks_demo/unity_catalog/demo_sales_operations/regional_sales"],
    soda_cloud_file_path="sc_config.yml",
)

print(result.get_logs_str())
```

The runner fetches the published contract from Soda Cloud and executes it against your data source.

{% hint style="info" %}
Calling verification in a loop? Call `configure_logging()` inside the loop, before each verification. The logging setup does not reset between iterations.
{% endhint %}

> Every parameter and result attribute: [Python API](/reference/soda-apis/python-api.md).
> {% endstep %}
> {% endstepper %}
> {% endtab %}
> {% endtabs %}

***

#### **Fail the pipeline when a check fails**

Verification is only worth putting in a pipeline if a failed check stops the run. Each tool reports the outcome in a way your orchestrator can act on.

{% tabs %}
{% tab title="Soda CLI" %}
`sodacli contract verify` returns:

| Code | Meaning                                       |
| ---- | --------------------------------------------- |
| `0`  | All checks passed.                            |
| `1`  | One or more checks failed. Fail the pipeline. |
| `2`  | Execution error. Retry or alert.              |
| `3`  | Authentication error. Check credentials.      |

A GitHub Actions step stops on a non-zero code with no extra wiring:

```yaml
- name: Verify data contract
  run: |
    sodacli auth login --host cloud.soda.io \
      --api-key-id ${{ secrets.SODA_KEY_ID }} \
      --api-key-secret ${{ secrets.SODA_KEY_SECRET }} \
      --no-interactive
    sodacli contract verify contracts/regional_sales.yml --no-interactive
```

{% endtab %}

{% tab title="Core CLI" %}
`soda contract verify` returns a different set of codes to `sodacli`. The same codes apply with `--use-runner`.

| Code | Meaning                                                                                                                      |
| ---- | ---------------------------------------------------------------------------------------------------------------------------- |
| `0`  | All checks passed.                                                                                                           |
| `1`  | One or more checks failed.                                                                                                   |
| `2`  | One or more checks warned and none failed.                                                                                   |
| `3`  | The verification could not run: a contract parse error, an engine error, or a scan that errored, was canceled, or timed out. |
| `4`  | Results could not be sent to Soda Cloud.                                                                                     |

Treat `2` deliberately. A warning is not a failure, so decide whether your pipeline continues on it:

```bash
soda contract verify -ds ds_config.yml -c contract.yaml --publish
status=$?
if [ $status -eq 1 ] || [ $status -ge 3 ]; then
  exit $status
fi
```

{% endtab %}

{% tab title="Python API" %}
Verification returns a `ContractVerificationSessionResult`. Read the outcome from it rather than parsing the logs:

```python
if not result.is_ok:
    print(result.get_errors_str())
    raise SystemExit(1)
```

| Attribute    | True when                                                               |
| ------------ | ----------------------------------------------------------------------- |
| `is_passed`  | No contract has failed checks.                                          |
| `is_failed`  | At least one contract has a failed check. Execution errors are ignored. |
| `is_warned`  | At least one contract has a warned check.                               |
| `has_errors` | At least one contract hit an execution error.                           |
| `is_ok`      | No failed checks and no errors.                                         |

For per-check detail, iterate `result.contract_verification_results` and read each `check_results` entry. {% endtab %} {% endtabs %}
{% endtab %}
{% endtabs %}

<i class="fa-hexagon-check" style="color:$success;">:hexagon-check:</i> **Your contract now runs wherever your data is produced.**

<p align="center">Prefer to try this in a notebook first? <a href="https://colab.research.google.com/drive/1zkV_2tLJ4ohdzmKGS3LgdFDDnTNTUXew?usp=sharing" class="button secondary medium" data-icon="up-right-from-square">Contract verification example notebook</a></p>

***

### What’s next?

<table data-view="cards"><thead><tr><th data-type="content-ref"></th><th></th></tr></thead><tbody><tr><td><a href="/data-observability/profiling.md">Profiling</a></td><td>Explore <strong>Profiling</strong> in the <strong>Discover</strong> tab to curate column selections for deeper analysis.</td></tr><tr><td><a href="/reference/soda-apis/rest-api/notification-rules.md">Notification Rules</a></td><td>Set up <strong>Notification Rules</strong> (bell icon → <strong>Add Notification Rule</strong>) to push alerts to Slack, Jira, PagerDuty, etc.</td></tr><tr><td><a href="/data-observability/metric-monitoring-dashboard/custom-monitors.md">Custom monitors</a></td><td>Dive into <strong>Custom</strong> <strong>Monitors</strong> via <code>scan.yml</code> or the UI for even more tailored metrics.</td></tr></tbody></table>

***

{% hint style="info" %}
You are **not logged in to Soda** and are viewing the default public documentation. Learn more about [Licensing & documentation access](/reference/documentation-access-and-licensing.md).

If you do have a Soda license, make sure to **log in to Soda Cloud in this same browser**.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.soda.io/quickstart.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.