> For the complete documentation index, see [llms.txt](https://docs.soda.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.soda.io/data-observability/metadata-data-sources.md).

# Metadata data sources

## Supported data sources

Below, find a snapshot of all the capabilities supported for each data source, including **metadata** and **metadata history**:

<table><thead><tr><th width="185.79998779296875">Data source</th><th width="171.20001220703125">Onboarding</th><th>Metadata</th><th>Metadata history</th><th>Querying data</th></tr></thead><tbody><tr><td><a data-mention href="/pages/ctP9k7n5AxdoMIBmXxD2">/pages/ctP9k7n5AxdoMIBmXxD2</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/xlK0ugO80HSregboFQUM">/pages/xlK0ugO80HSregboFQUM</a></td><td>✅</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td><a data-mention href="/pages/jggdwLvBUVJjJlkdmGUe">/pages/jggdwLvBUVJjJlkdmGUe</a></td><td>✅</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td><a data-mention href="/pages/qQQSBTgs2jlSAdkxg3mF">/pages/qQQSBTgs2jlSAdkxg3mF</a></td><td>✅</td><td>Upcoming</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/dESWjWE03JmceyspJhlb">/pages/dESWjWE03JmceyspJhlb</a> (<a href="/pages/eW38AlRtSFOjYNj9fD11">Pandas</a>, <a href="/pages/lA3k9OMWjmbLKD1OfPPA">Polars</a>, etc.)</td><td>Partial/Python only</td><td>—</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/eCdm3TE8crsXRy3LKZpO">/pages/eCdm3TE8crsXRy3LKZpO</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td>MySQL</td><td>Upcoming</td><td>—</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/yymUzQ59nkr9L3YddiK0">/pages/yymUzQ59nkr9L3YddiK0</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/xqU5pcfJVCubI3BL8MAH">/pages/xqU5pcfJVCubI3BL8MAH</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/AUAQ8kj87Q98kRNninUy">/pages/AUAQ8kj87Q98kRNninUy</a></td><td>✅</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td><a data-mention href="/pages/KxwupsjjHWfxegxEzjwS">/pages/KxwupsjjHWfxegxEzjwS</a></td><td>Partial/Python only</td><td>—</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/jmuttQi4HNCqZoA6ehA6">/pages/jmuttQi4HNCqZoA6ehA6</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/hnAwM7zDdGkq4RkDqhBt">/pages/hnAwM7zDdGkq4RkDqhBt</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/GzT8YXwoPxHYCfTPHap4">/pages/GzT8YXwoPxHYCfTPHap4</a></td><td>✅</td><td>✅</td><td>—</td><td>✅</td></tr><tr><td><a data-mention href="/pages/2aL0YaZUaMgSp9hJMghh">/pages/2aL0YaZUaMgSp9hJMghh</a></td><td>✅</td><td>—</td><td>—</td><td>✅</td></tr></tbody></table>

## Limitations & edge cases

Some data sources present **limitations** when it comes to **metadata** collection or retrieval. Keep in mind these caveats when choosing which data source(s) to connect to Soda.

### Oracle

* **Historical backfilling:** not possible.
* **Row count:** metadata row counts are calculated via `count(*)`. Soda does not use metadata for this metric in Oracle. It requires an additional package and/or is unreliable based on the schedule of that package.
* **Last modification time:** Soda uses metadata

Note that past data is only available for a limited amount of time, which varies depending on the system. The minimum goes back 120 h.

{% hint style="warning" %}
**Non-UTC timestamps are not recommended** when connecting Soda to Oracle data sources. Soda uses timezone data when available, but assumes UTC when the timezone is not provided by the data source.

Some databases convert timestamps to UTC, but **Oracle does not do any implicit conversions** and stores timestamps and timezone information as the user inputs them. Because of Oracle Python client limitations, all timezone information is stripped when Soda retrieves it, which means that Soda will read all timestamps as if they were UTC regardless of the original input.
{% endhint %}

***

### Postgres

Metadata is supported, but it requires some additional setup on Postgres's side.

* **Historical backfilling:** not possible.
* **Row count:** enabled out-of-the-box.
* **Last modification time:** `track_commit_timestamp` must be enabled: <https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-TRACK-COMMIT-TIMESTAMP>
  * If `track_commit_timestamp` is not enabled, Soda will return a warning.

***

### BigQuery

Metadata metrics are available and supported in BigQuery.

* **Historical backfilling:** possible.
* **Partition column:** can be suggested based on metadata available in BigQuery.
  * Soda will prioritize user-suggested columns.
  * If there are no user-suggested columns, Soda will try a metadata approach to find the partition column automatically.
  * If there are no columns found in the metadata of BigQuery, Soda will fall back on its own heuristic.

{% hint style="info" %}
**Partition column availability in BigQuery:**

If the user has configured a partitioning column on BigQuery's side, Soda will use it (given that it is a date/timestamp column).

Otherwise, Soda will fall back on a standard sampling method to detect the partition column.
{% endhint %}

***

### Redshift

* **Historical backfilling** is supported on Redshift and it is limited to 7 days for the metadata.
* **Modification time** does not include schema changes. Only:
  * inserts
  * updates
  * deletes

***

### Synapse

Synapse does not provide metadata history tables.

* **Historical backfilling:** not possible.
* **Last modification time:** not possible.
* **Row count:** current row counts are calculated via `count(*)`.\
  Soda does not use metadata for this metric in Synapse.
* **Quartile metrics** (Q1, median, Q3): not possible.\
  Synapse does not support quartile metrics.

***

{% if visitor.claims.plan === 'datasetStandard' %}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Dataset Standard license** documentation. Learn more about [Documentation access & licensing](/reference/documentation-access-and-licensing.md).
{% endhint %}
{% endif %}

{% if visitor.claims.plan === 'enterprise' %}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Team license** documentation. Learn more about [Documentation access & licensing](/reference/documentation-access-and-licensing.md).
{% endhint %}
{% endif %}

{% if visitor.claims.plan === 'enterpriseUserBased' %}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Enterprise license** documentation. Learn more about [Documentation access & licensing](/reference/documentation-access-and-licensing.md).
{% endhint %}
{% endif %}

{% if !(visitor.claims.plan === 'enterprise' || visitor.claims.plan === 'enterpriseUserBased' || visitor.claims.plan === 'datasetStandard') %}
{% hint style="info" %}
You are **not logged in to Soda** and are viewing the default public documentation. Learn more about [Documentation access & licensing](/reference/documentation-access-and-licensing.md).

If you do have a Soda license, make sure to **log in to Soda Cloud in this same browser**.
{% endhint %}
{% endif %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.soda.io/data-observability/metadata-data-sources.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.