Link Search Menu Expand Document

Integrate Soda with Metaphor

Last modified on 27-Sep-23

Integrate Soda with Metaphor to access details about the quality of your data from within the data catalog. The following video illustrates how to take advantage of the integration.

Prerequisites

  • You have a Soda Cloud account with Admin permissions, connected to an instance of Soda Library.
  • You have configured Soda Library to access the data source on which you want to run quality checks.
  • You have completed at least one Soda scan to validate that the data source‚Äôs datasets appear in Soda Cloud as expected.
  • You have a Metaphor account with the privileges necessary to allow you to add a data source.
  • You have a git repository in which to store the integration project files.

Set up the integration

  1. Sign into your Soda Cloud account and confirm that you see the datasets you expect to see in the data source you wish to test for quality.
  2. To connect your Soda Cloud account to your Metaphor account, create an .env file in your integration project in your git repo and include details according to the example below. To obtain the values for your Soda API keys, refer to Configure Soda Library.
     SODA_HOST=cloud.soda.i
     SODA_API_KEY_ID
     SODA_API_KEY_SECRET
     METAPHOR_ACCESS_KEY_ID
     METAPHOR_SECRET_ACCESS_KEY
     # s3 bucker path without trailing slash
     METAPHOR_S3_PATH
     SODA_LOGGING_LEVEL=INF
    

Run the integration

Contact Soda Support directly to acquire the assets and instructions to run the integration and view Soda Cloud details in your Metaphor catalog.

Use the integration

Access Soda Cloud to create agreements that execute checks against datasets in your data source each time you run a Soda scan manually, or orchestrate a scan using a data pipeline tool such as Airflow. Soda Cloud pushes data quality scan results to the corresponding data source in Metaphor so that users can review data quality information from within the catalog.

Refer to video above for details.

Go further


Was this documentation helpful?

What could we do to improve this page?

Documentation always applies to the latest version of Soda products
Last modified on 27-Sep-23