# Data source reference for Soda Core

Soda uses the official Python drivers for each supported data source. The configuration examples below include the default required fields, but you can extend them with any additional parameters supported by the underlying driver.

Each data source configuration must be written in a YAML file and passed as an argument using the CLI or Python API.

## General Guidelines

* Each configuration must include `type`, `name`, and a `connection` block.
* Use the exact structure required by the underlying Python driver.
* Test the connection before using the configuration in a contract.

```bash
soda data-source test -ds ds_config.yml
```

***

#### Connect to a data source already onboarded in Soda Cloud

If you have already [onboarded a data source](https://docs.soda.io/reference/broken-reference) in **Soda Cloud**, make sure to use the **exact same data source name** in your Soda Core data source configuration. This ensures that datasets are correctly identified and mapped to the existing data source in Soda Cloud, whether you run verifications locally with Soda Core or remotely via a Soda Runner.

***

#### Onboard a data source programmatically

It is possible to onboard a data source to Soda Cloud (and to a Soda Runner) after onboarding it using Soda Core.

> To learn how, head to [Broken link](https://docs.soda.io/reference/broken-reference "mention").

#### Using Environment Variables

You can reference environment variables in your data source configuration. This is useful for securely managing sensitive values (like credentials) or dynamically setting parameters based on your environment (e.g., dev, staging, prod).

**Example:**

<pre class="language-yaml" data-title="ds_config.yml"><code class="lang-yaml">type: postgres
name: postgres
connection:
  host: host_name
  port: 5432
  user: user_name
<strong>  password: ${env.SODA_DEMO_POSTGRES_PW}
</strong>  database: db_name
</code></pre>

Environment variables must be available in the runtime environment where Soda is executed (e.g., your terminal, CI/CD runner, or Docker container).

> For Soda to run quality scans on your data, you must configure it to connect to your data source.\
> To learn how to set up Soda from scratch and configure it to connect to your data sources, see Soda's [Quickstart](https://docs.soda.io/quickstart).

***

## Supported data sources

<table><thead><tr><th width="185.79998779296875">Data source</th><th width="171.20001220703125">Onboarding</th><th>Metadata</th><th>Metadata history</th><th>Querying data</th></tr></thead><tbody><tr><td><a data-mention href="data-source-reference-for-soda-core/amazon-athena">amazon-athena</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/bigquery">bigquery</a></td><td>✅</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/databricks">databricks</a></td><td>✅</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/dremio">dremio</a></td><td>✅</td><td>Upcoming</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/duckdb">duckdb</a> (<a href="data-source-reference-for-soda-core/pandas">Pandas</a>, <a href="data-source-reference-for-soda-core/polars">Polars</a>, etc.)</td><td>Partial/Python only</td><td>—</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/microsoft-fabric">microsoft-fabric</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td>MySQL</td><td>Upcoming</td><td>—</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/oracle">oracle</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/postgresql">postgresql</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/amazon-redshift">amazon-redshift</a></td><td>✅</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/spark-dataframe">spark-dataframe</a></td><td>Partial/Python only</td><td>—</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/snowflake">snowflake</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/sql-server">sql-server</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/synapse">synapse</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="data-source-reference-for-soda-core/trino">trino</a></td><td>✅</td><td>—</td><td>—</td><td>✅</td></tr></tbody></table>

<br>

***

{% if (visitor.claims.plan === 'datasetStandard')%}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Free license** documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}

{% if (visitor.claims.plan === 'enterprise')%}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Team license** documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}

{% if (visitor.claims.plan === 'enterpriseUserBased')%}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Enterprise license** documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}

{% if !(visitor.claims.plan === 'enterprise' || visitor.claims.plan === 'enterpriseUserBased' || visitor.claims.plan === 'datasetStandard')%}
{% hint style="info" %}
You are **not logged in to Soda** and are viewing the default public documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.soda.io/reference/data-source-reference-for-soda-core.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
