Databricks SQL

Access configuration details to connect Soda to a Databricks SQL data source.

For information about how to connect to Databricks Spark Dataframes, see: Example contract: Spark - Databricks

Connection configuration reference

Install the following package:

pip install -i https://pypi.cloud.soda.io/simple --pre -U "soda-databricks>4"

Data source YAML

Create the config file:

soda data-source create -f ds_config.yml

The data source configuration YAML should look like the following:

ds_config.yml

type: databricks
name: my_databricks
connection:
  host: ${env.DATABRICKS_HOST}
  http_path: ${env.DATABRICKS_SQL_HTTP_PATH}
  catalog: ${env.DATABRICKS_CATALOG}
  access_token: ${env.DATABRICKS_SQL_ACCESS_TOKEN}
  ## optional
  # session_configuration: <Optional session configuration dict>

Hive Metastore is supported. When the catalog is set to hive_metastore, Soda will use Hive-compatible queries.

Connection test

Test the data source connection:

soda data-source test -ds ds_config.yml

---

You are not logged in to Soda and are viewing the default public documentation. Learn more about Documentation access & licensing.