Link Search Menu Expand Document

Integrate Soda Cloud with Metaphor

Integrate Soda with Metaphor to access details about the quality of your data from within the data catalog. The following video illustrates how to take advantage of the integration.

Prerequisites

  • You have a Soda Cloud account with Admin permissions, connected to an instance of Soda Core.
  • You have configured Soda Core to access the data source on which you want to run quality checks.
  • You have completed at least one Soda scan to validate that the data source‚Äôs datasets appear in Soda Cloud as expected.
  • You have a Metaphor account with the privileges necessary to allow you to add a data source.
  • You have a git repository in which to store the integration project files.

Set up the integration

  1. Sign into your Soda Cloud account and confirm that you see the datasets you expect to see in the data source you wish to test for quality.
  2. To connect your Soda Cloud account to your Metaphor account, create an .env file in your integration project in your git repo and include details according to the example below. To obtain the values for your Soda API keys, refer to the Connect to Soda Cloud documentation.
SODA_HOST=cloud.soda.io
SODA_API_KEY_ID=
SODA_API_KEY_SECRET=

METAPHOR_ACCESS_KEY_ID=
METAPHOR_SECRET_ACCESS_KEY=
# s3 bucker path without trailing slash.
METAPHOR_S3_PATH=

SODA_LOGGING_LEVEL=INFO

Run the integration

Contact Soda Support directly to acquire the assets and instructions to run the integration and view Soda Cloud details in your Metaphor catalog.

Use the integration

Access Soda Cloud to create monitors that execute tests against datasets in your data source each time you run a Soda scan manually, or orchestrate a scan using a data pipeline tool such as Airflow. Soda Cloud pushes data quality scan results to the corresponding data source in Metaphor so that users can review data quality information from within the catalog.

Refer to video above for details.

Go further


Last modified on 01-Jul-22

Was this documentation helpful?
Share feedback in the Soda community on Slack.

Help improve our docs!

  • Request a docs change.
  • Edit this page in our GitHub repo.