# Soda Runner basic concepts

{% hint style="danger" %}
**Nomenclature change: "Agent" is now "Runner"**

The Soda Agent has been renamed to [**Soda Runner**](https://docs.soda.io/reference/soda-agent-basic-concepts) across all Soda products and interfaces. This change affects terminology throughout the platform, including:

* Soda Cloud UI labels
* API permission names: `MANAGE_DATASOURCES_AND_AGENTS` → `MANAGE_DATASOURCES_AND_RUNNERS`
* CLI flags: `--use-agent` → `--use-runner`
* Python API methods: `verify_contract_on_agent` → `verify_contract_on_runner`

If you are following older documentation or have existing scripts and automation that reference "agent", we recommend to gradually replace every occurrence of "agent" / "Agent" with "runner" / "Runner". The underlying functionality is unchanged.
{% endhint %}

The **Soda Runner** is a tool that empowers Soda Cloud users to securely access data sources to scan for data quality. For a self-hosted runner, create a Kubernetes cluster in a cloud services provider environment, then use Helm to deploy a Soda Runner in the cluster.

This setup enables Soda Cloud users to securely connect to data sources (Snowflake, Amazon Athena, etc.) from within the Soda Cloud web application. Any user in your Soda Cloud account can add a new data source via the runner, then write their own no-code checks to check for data quality in the new data source.

What follows is an extremely abridged introduction to a few basic elements involved in the deployment and setup of a self-hosted Soda Runner.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FtgiH9kAinMQP5YfA0u8Q%2Fhttps___content.gitbook.com_content_oV0A6Eua8LUIyWgHxsjf_blobs_0eaZC4imQu27Et6yROAd_agent-diagram.avif?alt=media&#x26;token=d2d92f8f-bc66-48f3-9857-aa0f45c120aa" alt=""><figcaption></figcaption></figure>

### What is Soda Library?

**Soda Library** is a **Python library** and command-line tool that serves as the backbone of Soda technology. It is the software that performs the work of converting user-defined input into SQL queries that execute when you run scans for data quality in a data source. Connect Soda Library to a **Soda Cloud** account where you and your team can use the web application to collaborate on data quality monitoring.

### What is Soda Cloud?

**Soda Cloud** is Soda's **web application**. It provides a friendly, intuitive and low-barrier interface that can be used by technical and non-technical users alike. Soda Cloud allows for collaboration between teams and for different permission levels depending on your organization's structure. In the web application, for example, non-technical users can propose new checks and contract changes, while engineering teams can review the suggestions and approve or modify them.

### What are data contracts?

**Data contracts** are **formal agreements** between your organization's teams (usually between data producers and data consumers), and **they specify the expected shape, quality, and behavior of a dataset**. A data contract sets clear rules for what the data must look like and how it should perform, so that producers and consumers of data share a common understanding.

Both **Soda Library** and **Soda Cloud** make use of data contracts written in **YAML language**. Contracts include **checks** for data quality. The checks are tests that Soda Library executes when it runs a scan of your data.

> Go to [contract-language-reference](https://docs.soda.io/reference/contract-language-reference "mention") to learn more about contract structure and requirements.

### What is Soda Runner?

**Soda Runner** is essentially Soda Library functionality that you deploy in a Kubernetes cluster in your own cloud services provider environment. When you deploy a runner, you also deploy two types of workloads in your Kubernetes cluster from a Docker image:

* a **Soda Runner Orchestrator** which creates Kubernetes Jobs to trigger scheduled and on-demand scans of data
* a **Soda Runner Scan Launcher** which wraps around Soda Library, the tool which performs the scan itself

### How does Soda integrate with Kubernetes?

**Kubernetes** is a system for orchestrating containerized applications; a **Kubernetes cluster** is a set of resources that supports an application deployment.

You need a Kubernetes cluster in which to deploy the containerized applications that make up the **Soda Runner**. Kubernetes uses the concept of [**Secrets** ](https://kubernetes.io/docs/concepts/configuration/secret/)that the Soda Runner Helm chart employs to store connection secrets that you specify as values during the Helm release of the Soda Runner. Depending on your cloud provider, you can arrange to store these Secrets in a specialized storage such as [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/basic-concepts) or [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html).

> Learn more about [using external secrets](https://docs.soda.io/deployment-options/soda-agent/upgrade-soda-agent/upgrade-guide/deploy-a-self-hosted-soda-agent-from-sodas-private-container-registry#using-existing-external-secrets).

The Jobs that the runner creates access these Secrets when they execute.

> Learn more about [Kubernetes concepts](https://www.youtube.com/watch?v=BOj1sgWVXko).

### Where can a Soda Runner be deployed?

Within a cloud services provider environment is *where* you create your Kubernetes cluster. You can deploy a Soda Runner in **any environment in which you can create Kubernetes clusters**, such as:

* Amazon Elastic Kubernetes Service (EKS)
* Microsoft Azure Kubernetes Service (AKS)
* Google Kubernetes Engine (GKE)
* Any Kubernetes cluster version 1.21 or greater which uses standard Kubernetes
* Locally, for testing purposes, using tools like [Minikube](https://minikube.sigs.k8s.io/docs/), [microk8s](https://microk8s.io/docs), [kind](https://kind.sigs.k8s.io/), [k3s](https://docs.k3s.io/), or [Docker Desktop](https://www.docker.com/products/docker-desktop/) with Kubernetes support.

### What is Helm?

**Helm** is a **package manager for Kubernetes** which bundles YAML files together for storage in a public or private repository. This bundle of YAML files is referred to as a **Helm chart**. <mark style="background-color:$primary;">The Soda Runner is a Helm chart</mark>. Anyone with access to the Helm chart’s repo can deploy the chart to make use of YAML files in it.

> Learn more about [Helm concepts](https://www.youtube.com/watch?v=-ykwb1d0DXU).

The Soda Runner Helm chart is stored on a [public repository](https://helm.soda.io/soda-agent/) and published on [ArtifactHub.io](https://artifacthub.io/packages/helm/soda-agent/soda-agent). Anyone can use Helm to find and deploy the Soda Runner Helm chart in their Kubernetes cluster.

### Why Kubernetes?

Kubernetes is the most **powerful** and **future-proof** platform for running the Soda Runner because it delivers the best of both worlds: the **flexibility of raw compute** without the operational burden, and the **scalability of managed services** without their restrictions.

* Kubernetes goes far beyond raw compute like EC2 or traditional Virtual Machines (VMs) by **abstracting away the heavy lifting of networking, deployments, and scaling**, while still giving teams precise control when needed. Practically, this makes it easy for Soda’s customers to [deploy, manage, and upgrade Soda Runners](https://docs.soda.io/deployment-options/soda-agent/deploy-soda-agent) using [Kubernetes](#how-does-soda-integrate-with-kubernetes) and [Helm](#what-is-helm), always staying up to date with the latest releases.
* Unlike fully managed options such as AWS Lambda, **Kubernetes has no execution time limits** and is built to handle long-running, stateful, and highly scalable workloads. This means **Soda is not limited to lightweight samples** but can perform complete, row-level operations—powering advanced capabilities like Diagnostics Warehouse, which securely stores the exact failing records inside your own infrastructure, and [Reconciliation Checks](https://docs.soda.io/reference/contract-language-reference/reconciliation-checks), which compare data at row-level across sources.

Whether running in the cloud or on-premises, Kubernetes ensures resilience, portability, and cost-efficient resource use, making it the clear choice for complex, enterprise-grade data quality workloads.

***

### Go further <a href="#go-further" id="go-further"></a>

* [deploy-a-soda-agent-in-a-kubernetes-cluster](https://docs.soda.io/deployment-options/soda-agent/deploy-soda-agent/deploy-a-soda-agent-in-a-kubernetes-cluster "mention")

<br>

***

{% if (visitor.claims.plan === 'datasetStandard')
%}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Free license** documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}

{% if (visitor.claims.plan === 'enterprise')
%}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Team license** documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}

{% if (visitor.claims.plan === 'enterpriseUserBased')
%}
{% hint style="success" %}
You are **logged in to Soda** and seeing the **Enterprise license** documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}

{% if !(visitor.claims.plan === 'enterprise' || visitor.claims.plan === 'enterpriseUserBased' || visitor.claims.plan === 'datasetStandard')
%}
{% hint style="info" %}
You are **not logged in to Soda** and are viewing the default public documentation. Learn more about [documentation-access-and-licensing](https://docs.soda.io/reference/documentation-access-and-licensing "mention").
{% endhint %}
{% endif %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.soda.io/reference/soda-agent-basic-concepts.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.