Databricks SQL

Access configuration details to connect Soda to a Databricks SQL data source.

For information about how to connect to Databricks Spark Dataframes, see: Example contract: Spark - Databricks

Connection configuration reference

Install the following package:

pip install soda-databricks

Data source YAML

Core connection

Create the config file:

soda data-source create -f ds_config.yml

Databricks supports three authentication methods. Set auth_type to choose one. If you omit auth_type, Soda uses personal access token authentication.

Authentication method	Soda Cloud support	Soda Core support
Personal access token	✅	✅
Databricks OAuth (machine-to-machine)	✅	✅
Azure service principal	✅	✅

Every method shares these connection parameters:

Parameter	Required	Default	Description
type	Yes	—	Must be databricks.
host	Yes	—	Databricks workspace hostname, for example abc.cloud.databricks.com. A leading https:// or http:// is removed.
http_path	Yes	—	HTTP path for the SQL warehouse or cluster.
catalog	No	—	Default catalog to use.
session_configuration	No	—	Optional session configuration dictionary.
auth_type	No	personal-access-token	Authentication method. One of personal-access-token, databricks-oauth-m2m, azure-service-principal.

Hive Metastore is supported. When the catalog is set to hive_metastore, Soda will use Hive-compatible queries.

Personal access token

Supported in Soda Cloud and Soda Core.

ds_config.yml

type: databricks
name: my_databricks
connection:
  host: ${env.DATABRICKS_HOST}
  http_path: ${env.DATABRICKS_SQL_HTTP_PATH}
  catalog: ${env.DATABRICKS_CATALOG}

  # auth_type: personal-access-token   # optional; the default when omitted
  access_token: ${env.DATABRICKS_SQL_ACCESS_TOKEN}

  ## optional
  # session_configuration: <session configuration dict>

Parameter	Required	Default	Description
auth_type	No	personal-access-token	Omit, or set to personal-access-token.
access_token	Yes	—	Databricks personal access token.

---

Databricks OAuth (machine-to-machine)

Supported in Soda Cloud and Soda Core.

Authenticates with a Databricks-managed OAuth service principal using the client-credentials flow. Create the service principal and generate an OAuth secret in Databricks (see Databricks' documentation on OAuth machine-to-machine (M2M) authentication). Soda derives the token endpoint from host, so no token URL is required.

ds_config.yml

type: databricks
name: my_databricks
connection:
  host: ${env.DATABRICKS_HOST}
  http_path: ${env.DATABRICKS_SQL_HTTP_PATH}
  catalog: ${env.DATABRICKS_CATALOG}

  auth_type: databricks-oauth-m2m
  client_id: ${env.DATABRICKS_CLIENT_ID}
  client_secret: ${env.DATABRICKS_CLIENT_SECRET}

Parameter	Required	Default	Description
auth_type	Yes	—	Must be databricks-oauth-m2m.
client_id	Yes	—	Databricks OAuth service-principal client ID.
client_secret	Yes	—	Databricks OAuth service-principal client secret.

---

Azure service principal

Supported in Soda Cloud and Soda Core.

Authenticates Azure Databricks with a Microsoft Entra ID (Azure AD) service principal. Register the application in Entra ID and note its client ID, client secret, and directory (tenant) ID (see Microsoft's documentation on creating a Microsoft Entra ID application and service principal). Soda requests the token from login.microsoftonline.com, so the tenant ID is required.

ds_config.yml

type: databricks
name: my_databricks
connection:
  host: ${env.DATABRICKS_HOST}
  http_path: ${env.DATABRICKS_SQL_HTTP_PATH}
  catalog: ${env.DATABRICKS_CATALOG}

  auth_type: azure-service-principal
  azure_client_id: ${env.DATABRICKS_AZURE_CLIENT_ID}
  azure_client_secret: ${env.DATABRICKS_AZURE_CLIENT_SECRET}
  azure_tenant_id: ${env.DATABRICKS_AZURE_TENANT_ID}

Parameter	Required	Default	Description
auth_type	Yes	—	Must be azure-service-principal.
azure_client_id	Yes	—	Entra ID (Azure AD) service-principal application/client ID.
azure_client_secret	Yes	—	Entra ID (Azure AD) service-principal client secret.
azure_tenant_id	Yes	—	Entra ID (Azure AD) directory/tenant ID.

---

Test the connection

Test the data source connection:

soda data-source test -ds ds_config.yml

---

You are not logged in to Soda and are viewing the default public documentation. Learn more about Licensing & documentation access.

If you do have a Soda license, make sure to log in to Soda Cloud in this same browser.