Link Search Menu Expand Document

Examine failed rows

When a scan results in a failed test, Soda Cloud displays details of the scan results in each monitor’s Monitor History view. To offer more insight into the data that failed a test during a scan, Soda Cloud can display failed rows in a monitor’s history.

failed-rows

Use a missing-value Metric Type to collect failed row samples

If you use one of the following Metric Types in a test that you define in a monitor, Soda SQL automatically sends a sample of five failed rows associated with the failed test to Soda Cloud with the scan results.

  • Missing Values
  • Invalid Values
  • Distinct

When Soda Cloud runs its next scheduled scan of your dataset, it collects and displays a sample of failed rows for the monitors that use the above-listed metric types.

If you are a Soda SQL user and have connected to your Soda Cloud account, you can add configurations to your scan YAML file to explicitly send failed row samples to Soda Cloud. See Send failed rows to Soda Cloud for instructions.

Disable failed row samples

Where your datasets contain sensitive or private information, you may not want to send failed row samples from your data source to Soda Cloud. In such a circumstance, you can disable the feature completely in Soda Cloud.

To prevent Soda Cloud from receiving any sample data or failed row samples for any datasets in any data sources to which you have connected your Soda Cloud account, proceed as follows:

  1. As an Admin, log in to your Soda Cloud account and navigate to your avatar > Organization Settings.
  2. In the Company tab, check the box to “Disable storage of sample data and failed row samples in Soda Cloud.”, then Save.

If you use Soda SQL to programmatically schedule scans of individual datasets, you can configure Soda SQL to send a dataset’s failed row samples to a secure location within your organization’s infrastructure, such as an Amazon S3 bucket or Google Big Query. Refer to Reroute failed row samples for details.

Go further



Last modified on 18-Oct-21

Was this documentation helpful?
Give us your feedback in the #soda-docs channel in the Soda community on Slack or open an issue in GitHub.