> For the complete documentation index, see [llms.txt](https://docs.soda.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.soda.io/soda-documentation/soda-ai/bring-your-own-key-byok.md).

# Bring your own key (BYOK)

Learn how to run the Soda AI features on your own account with OpenAI, Anthropic, Google Gemini, Azure OpenAI or Portkey

{% hint style="info" %}
Bring your own key is **only available in organizations fully on Soda v4**.

[Migrate to Soda v4](https://docs.soda.io/soda-documentation/govern/migrate-from-v3-to-v4.md) to have access to the newest Soda capabilities.
{% endhint %}

Bring your own key (BYOK) runs the Soda AI features on an account that your organization holds with a model provider. Soda supports these provider types:

* OpenAI
* Anthropic
* Google Gemini, through the Gemini API or Vertex AI
* Azure OpenAI (Microsoft Foundry)
* Portkey AI Gateway

With a provider of your own:

* **Requests go to your provider account.** Soda sends the requests of the Soda AI Agent and Contract Autopilot to the account behind your key. Your agreement with the provider applies, together with the retention, privacy and regional settings of that account. The provider bills your organization for the usage.
* **You keep control of the key.** Soda stores a reference to the key and never the key itself. The reference points to an environment variable on your runner or to a Soda Cloud secret. You rotate or revoke the key in the system that holds it.

> Learn what the AI features send to the provider in [AI security, compliance & privacy](https://docs.soda.io/soda-documentation/soda-ai/ai-security-compliance-and-privacy.md).

## AI configuration

Admins manage providers in the **AI** tab of **Organization settings**.

### Organization settings - AI tab visibility

The **AI** tab shows when BYOK is available to your organization and at least one AI feature is available to it. If the tab is missing, contact <support@soda.io>.

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FGCCnH1SC9VGkauiHdk22%2Fimage.png?alt=media&amp;token=62726a0c-31a4-4096-a013-0b7e04ee91af" alt=""><figcaption></figcaption></figure>

The tab has two parts:

* The **Providers** list. It shows each provider with its type, its model, who added it, who made it active, and its status.
* The **Soda AI** box. It holds the switches that turn the AI features on or off.

### Providers

One provider is **Active** at a time. The Soda AI Agent and Contract Autopilot both run on the active provider. Every other provider has the status **Available**.

#### **Soda's built-in provider**

The **Soda (OpenAI)** row is the OpenAI account that Soda operates. It shows only in organizations that may use Soda's key. In those organizations it:

* is the active provider until you make another provider active,
* is included in your Soda subscription,
* cannot be edited or removed, and
* has no settings. Soda chooses the model.

In an organization without the **Soda (OpenAI)** row, the AI features stay off until you add a provider.

#### **Add a provider**

{% stepper %}
{% step %}
Click **Add provider**. The **Provider configuration** panel opens.
{% endstep %}

{% step %}
The first time, read the notice **Before you add your own key** and click **Agree and continue**. Soda records your organization, your user and the time.
{% endstep %}

{% step %}
Choose the provider type.
{% endstep %}

{% step %}
Enter a **Name**. The name identifies the provider in the list, for example `Team OpenAI`.
{% endstep %}

{% step %}
Fill in the fields of the provider type. The sections below list them.
{% endstep %}

{% step %}
Click **Save**.
{% endstep %}
{% endstepper %}

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FmuxZiz4VbfpXhhRxMcK4%2Fimage.png?alt=media&amp;token=05961465-3689-42f0-94d6-61430a8cd060" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}
**A new provider becomes the active provider when you save it.** The Soda AI Agent and Contract Autopilot use it from their next request.
{% endhint %}

Every key field takes a reference, not the key itself. See [API key reference formats](#api-key-reference-formats).

**OpenAI**

* **API key**: a reference to an OpenAI API key.
* **Model**: optional. Contract Autopilot runs this model. Leave it empty to run the runner's default model. The Soda AI Agent ignores this field: on an OpenAI provider it runs the model that Soda chooses.

{% hint style="warning" %}
**The Soda AI Agent does not support OpenAI projects with zero data retention.** It continues a conversation from the responses that OpenAI stores, and zero data retention turns that storage off.
{% endhint %}

**Anthropic**

* **API key**: a reference to an Anthropic API key. Use a key scoped to a workspace. Anthropic rejects keys created at the organization level.
* **Model**: optional. The Soda AI Agent and Contract Autopilot both run this model. Leave it empty to run the default model of each. The hint under the field names the default model of the Soda AI Agent.

Soda sends requests to the Anthropic API directly. A custom base URL, such as a gateway in front of Anthropic, is not supported.

**Google Gemini**

Choose how Soda authenticates in **Authentication**:

* **Vertex AI service account**
  * **Service account key**: paste the JSON key of the service account into a Soda Cloud secret, then reference the secret here. The service account needs the **Vertex AI User** role.
  * **Project ID**: optional. Leave it empty to use the project in the key file.
  * **Location**: **Global**, **EU (multi-region)** or **US (multi-region)**.
* **Gemini API key**
  * **API key**: a reference to a Gemini API key. Use a key from a project with billing enabled. With a free-tier key, Google can use your prompts to improve its products.

For both options:

* **Model**: optional. The Soda AI Agent and Contract Autopilot both run this model. Leave it empty to run the default model of each. The hint under the field names the default model of the Soda AI Agent.

**Azure OpenAI (Microsoft Foundry)**

* **Endpoint**: the endpoint of your Azure OpenAI or Microsoft Foundry resource. Soda accepts `https://<resource>.openai.azure.com`, `https://<resource>.services.ai.azure.com` and `https://<resource>.cognitiveservices.azure.com`, each optionally followed by `/openai/v1`. Other hosts, such as a gateway in front of the resource, are not accepted.
* **Authentication**:
  * **API key**
    * **API key**: paste one of the keys of the resource into a Soda Cloud secret, then reference the secret here.
  * **Entra ID service principal**
    * **Tenant ID**: the directory (tenant) ID, or one of the domains of the tenant, such as `contoso.onmicrosoft.com`.
    * **Client ID**: the application (client) ID of the service principal.
    * **Client secret**: a reference to the client secret of the service principal. The service principal needs the **Cognitive Services OpenAI User** role on the resource.
* **Deployment**: required. Enter the name of the deployment on your Azure resource, not the name of the model. The Soda AI Agent and Contract Autopilot both call this deployment.

Requirements for the Soda AI Agent:

* The deployment must serve a reasoning model. A deployment of a model without reasoning fails.
* If the resource restricts network access, allow Soda's IP addresses on it.
* The resource stores each response, and the Soda AI Agent continues a conversation from the stored responses. Soda has no setting that turns this storage off.

**Portkey AI Gateway**

* **Provider behind the gateway**: the provider behind the slug of your model string in the Portkey model catalog.
  * **OpenAI**: OpenAI itself.
  * **Anthropic**: the Anthropic API itself.
  * **Other (OpenAI-compatible)**: every other provider. This includes Claude on Amazon Bedrock or Vertex AI, and Azure OpenAI.
* **API key**: paste your Portkey API key into a Soda Cloud secret, then reference the secret here. Portkey holds the key of the provider behind it. Soda never receives that key.
* **Model string**: required. Enter the model string from your Portkey model catalog in the form `@provider-slug/model-name`, for example `@openai-prod/gpt-5-mini`. The Soda AI Agent and Contract Autopilot both run this model.

Soda refuses a response that comes from a provider other than the one you chose in **Provider behind the gateway**.

#### **API key reference formats**

A key field accepts one of two reference formats: an environment variable or a Soda Cloud secret. Soda resolves the reference at run time and never writes the key to Soda Cloud's storage. The field refuses any other value, including the key itself.

**1. Environment variable: `${env.NAME}`**

Use the `${env.NAME}` format when the key is set as an environment variable on a self-hosted runner.

```bash
${env.LLM_API_KEY}
```

At run time, the runner reads the environment variable. The key stays in your own infrastructure and never enters Soda Cloud.

Use this format when:

* you already **manage credentials through the environment of your platform**, for example Kubernetes secrets mounted as environment variables, or
* you want the key to **stay outside Soda Cloud**.

{% hint style="info" %}
An environment variable does not work for the Soda AI Agent. The Soda AI Agent runs in Soda Cloud, where you cannot set environment variables. Use a Soda Cloud secret if the provider also runs the Soda AI Agent.
{% endhint %}

**2. Soda Cloud secret: `${secret.NAME}`**

Use the `${secret.NAME}` format to keep the key in Soda Cloud's secrets manager. The secret must exist under **Organization settings → Secrets**. In the form, the **Secret** button on the field inserts the reference.

```
${secret.llm-api-key}
```

At run time, Soda reads the secret from the encrypted secret store of your organization.

Use this format when:

* the provider runs the Soda AI Agent,
* you **do not run a self-hosted runner**, or
* you want to **manage every credential in one place**.

{% hint style="info" %}
**A `${secret.…}` reference links the provider to the secret.** While a provider references a secret, Soda Cloud prevents the deletion of that secret. The deletion dialog lists the providers to update first.
{% endhint %}

#### Manage existing providers

Each provider that you added has a three-dot menu:

* **Make active**: makes the provider the active provider. For an Azure OpenAI or Portkey provider, the panel opens first and asks for the deployment or the model string.
* **Edit**: changes the name, the key reference and the other fields. For the active provider, it also changes the model. The provider type cannot change. To use another type, add a new provider.
* **Remove**: removes the provider after you confirm.

If you remove the active provider, the AI features go back to **Soda (OpenAI)** when your organization has that row. Without that row, the AI features have no provider until you make another provider active.

### What runs on the active provider

<figure><img src="https://1123167021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FA2PmHkO5cBgeRPdiPPOG%2Fuploads%2FtG8PlHICzybGcb4dsP4o%2Fimage.png?alt=media&amp;token=9f53f321-3d9b-43ae-85fa-8f350faa449a" alt=""><figcaption></figcaption></figure>

The active provider runs two capabilities:

* **Soda AI** runs the **Soda AI Agent**, the chat in Soda Cloud.
* **Runner** runs **Contract Autopilot**. A runner makes these requests: the Soda-hosted runner or a self-hosted runner.

The model that each capability runs depends on the provider type:

| Provider type      | Field in the form          | Soda AI Agent                                      | Contract Autopilot                               |
| ------------------ | -------------------------- | -------------------------------------------------- | ------------------------------------------------ |
| OpenAI             | **Model**, optional        | The model that Soda chooses. The field is ignored. | The model in the field, or the runner's default. |
| Anthropic          | **Model**, optional        | The model in the field, or Soda AI's default.      | The model in the field, or the runner's default. |
| Google Gemini      | **Model**, optional        | The model in the field, or Soda AI's default.      | The model in the field, or the runner's default. |
| Azure OpenAI       | **Deployment**, required   | The deployment in the field.                       | The deployment in the field.                     |
| Portkey AI Gateway | **Model string**, required | The model string in the field.                     | The model string in the field.                   |

The form has no field for the reasoning effort.

#### Runner support

Every runner supports an OpenAI provider. An Anthropic, Google Gemini, Azure OpenAI or Portkey provider needs a runner that supports that provider type. Contract Autopilot skips a dataset when its runner does not support the active provider. If that happens on a self-hosted runner, upgrade the runner.

#### Make a provider active

{% stepper %}
{% step %}
In the **Providers** list, open the three-dot menu of the provider.
{% endstep %}

{% step %}
Click **Make active**.
{% endstep %}

{% step %}
Confirm. For an Azure OpenAI or Portkey provider, the panel opens instead: enter the deployment or the model string and click **Save**.
{% endstep %}
{% endstepper %}

The Soda AI Agent and Contract Autopilot use the new active provider from their next request.

#### Switch back to the built-in provider

If your organization has the **Soda (OpenAI)** row:

{% stepper %}
{% step %}
Open the three-dot menu of **Soda (OpenAI)**.
{% endstep %}

{% step %}
Click **Make active**.
{% endstep %}

{% step %}
Confirm.
{% endstep %}
{% endstepper %}

#### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

Setup:

| Problem                                                                 | What to do                                                                                                                                                                                                      |
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| The **AI** tab is missing from Organization settings.                   | BYOK or the AI features are not available to your organization. Contact <support@soda.io>.                                                                                                                      |
| A key field shows "Must be a ${secret.NAME} or ${env.NAME} reference."  | Enter a `${env.NAME}` or `${secret.NAME}` reference. The field refuses a key pasted directly and every other format.                                                                                            |
| The **Endpoint** field of an Azure OpenAI provider refuses the URL.     | Enter the URL of the resource on `openai.azure.com`, `services.ai.azure.com` or `cognitiveservices.azure.com`, over `https`, with no port and no path other than `/openai/v1`.                                  |
| A `${secret.NAME}` reference does not resolve at run time.              | Make sure the secret exists under **Organization settings → Secrets** and that its name matches the reference exactly. Secret names are case-sensitive.                                                         |
| A `${env.NAME}` reference does not resolve at run time.                 | Make sure the environment variable is set on the self-hosted runner and that the name matches exactly. Environment variable names are case-sensitive. The Soda AI Agent cannot read your environment variables. |
| Contract Autopilot skips datasets after you change the active provider. | The runner of the data source does not support the provider type. Upgrade the self-hosted runner, or make a provider active that the runner supports.                                                           |

Messages in the Soda AI Agent chat. Each message can also quote the provider's own error text.

| Message                                                                                                                                 | What to do                                                                                                                                                                            |
| --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| "\<Provider> rejected your organization's API key."                                                                                     | The provider refused the key. Check that the key is valid and not revoked, then update the secret or the key reference.                                                               |
| "Azure OpenAI refused your organization's API key."                                                                                     | Check that the key belongs to the resource at the configured endpoint. If the resource restricts network access, allow Soda's IP addresses.                                           |
| "Azure OpenAI refused your organization's Entra ID service principal."                                                                  | Check the tenant ID, the client ID and the client secret. Check that the service principal has the **Cognitive Services OpenAI User** role on the resource, and the network rules.    |
| "The model configured for your organization is not available with its AI provider API key."                                             | The provider does not know the model, or your account cannot use it. Edit the provider and correct the **Model**.                                                                     |
| "The deployment configured for your organization was not found on its Azure OpenAI resource, or it does not serve a reasoning model."   | Edit the provider. Check the **Deployment** name and the **Endpoint**, and use a deployment of a reasoning model.                                                                     |
| "Portkey could not serve the model configured for your organization"                                                                    | Check the **Model string** and **Provider behind the gateway**. The slug must exist in your Portkey catalog, and the provider behind it must be the one you chose.                    |
| "Your organization's \<Provider> account has no remaining quota."                                                                       | The quota or the budget of the provider account is used up. Raise it in the provider account.                                                                                         |
| "We couldn't decrypt your organization's stored \<Provider> key." or "We couldn't securely access your organization's \<Provider> key." | Soda could not read the key. Check that the secret still exists, then contact <support@soda.io>.                                                                                      |
| "This conversation has reached its length limit. Start a new conversation to continue."                                                 | Start a new conversation. See [Unstructured input](https://docs.soda.io/soda-documentation/soda-ai/soda-ai-agent/unstructured-input.md#size-limits) for the effect of attached files. |

<br>

***

{% hint style="info" %}
You are **not logged in to Soda** and are viewing the default public documentation.

Learn more about [Licensing & documentation access](https://docs.soda.io/soda-documentation/govern/security-licensing-and-compliance/documentation-access-and-licensing.md) or <a href="https://docs.soda.io/~gitbook/auth/login" class="button primary small">Log in to Soda Cloud</a>.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.soda.io/soda-documentation/soda-ai/bring-your-own-key-byok.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
