Sign up for a free Soda Cloud account, connect Soda SQL to your account, integrate with Slack, then create a monitor and an alert to begin monitoring your data.
- If you have not already done so, create a free Soda Cloud account at cloud.soda.io. You, as the first user in your organization to sign up for a Soda Cloud account, become the account’s Admin by default. Learn more about Roles and rights in Soda Cloud. An Admin role is necessary to complete this tutorial.
- If you have not already done so, follow the instructions to Install and Configure Soda SQL in your local environment.
- Connect your Soda Cloud account to your existing Soda SQL instance.
- Run a scan from Soda SQL so that it pushes dataset and test results to your Soda Cloud account.
Connect your Soda Cloud account to your Slack workspace. Making this connection enables you to send Slack notifications to your team when a data issue triggers an alert.
If you do not use Slack, Soda Cloud notifies you and any teammates you invite via email.
- In Soda Cloud, navigate to your avatar > Organization Settings > Integrations, then follow the guided steps to authorize Soda Cloud to connect to your Slack workspace.
- Select the all Slack channels to which you might send notifications when Soda finds an issue with your data, then Save.
After Soda Cloud completes its first scheduled scan of your data source, you can use the data and metadata it collected, such as column names and data types, to create a monitor and alert.
Note that Soda Cloud also automatically created a row count anomaly detection monitor and for each dataset that contains time-series data, and a schema evaluation monitor for each dataset. The anomaly detection monitor enables Soda Cloud to start learning row count patterns in your dataset over the course of the next few scheduled scans and surface anything it recognizes as anomalous. See anomaly detection for details. The schema evaluation monitor notifies you when columns in a dataset have been added, removed, or changed since the previous Soda scan of the dataset. See Monitor schema evolution for details.
For a new monitor, you define several details including which data to test, what tests to run, and whom to notify when bad data triggers an alert.
- In Soda Cloud, navigate to the Monitors dashboard, then click the stacked dots to Create Monitor. Select the type
Metric, then follow the guided steps to complete the setup. Use the following input values for reference.
- Metric Type:
(For datasets you added via Soda Cloud, you can only select Row Count for this field. Soon, Soda Cloud will make more Metric Types available for selection for all datasets.)
- Column: n/a
- Evaluation type:
- Critical Alert:
if less than;
- Add people, roles or channels to alert:
your slack channel, if using Slack
- Notify about:
- When Soda SQL runs its next scan of your data source, it runs the test you just created in your monitor. If the test fails, the failure triggers the alert you defined and sends a notification to the Slack channel you identified in your monitor, or your email address if you do not use Slack.
Refer to Create monitors and alerts for further details.
When Soda SQL completes a scan of your data source, it runs your test and presents the results in the Monitors dashboard.
- Review the results of your test in the Monitor Results table in Soda Cloud to find the result for the monitor you just created. See the example below in which a test passed.
- Click the monitor result to access details that can help you diagnose and solve the data issue.
- Check your Slack channel or email inbox; if the test failed, the scan surfaced a data issue that triggered your alert so Soda Cloud sent a notification.
- Learn more about Soda Cloud Architecture.
- Use Soda Cloud to automatically detect anomalies in your data.
- Find out how to examine failed rows for tests that failed.
- Need help? Join the Soda community on Slack.
Last modified on 26-Nov-21