Use a for each configuration to execute checks against multiple datasets during a scan.
for each dataset T: datasets: - dim_products% - fact% - exclude fact_survey_response checks: - row_count > 0
Add a for each section to your checks YAML file to specify a list of checks you wish to execute on multiple datasets.
- Add a
for each dataset Tsection header anywhere in your YAML file. The purpose of the
Tis only to ensure that every
for eachconfiguration has a unique name.
- Nested under the section header, add two nested keys, one for
datasetsand one for
- Nested under
datasets, add a list of datasets against which to run the checks. Refer to the example below that illustrates how to use
excludeconfigurations and wildcard characters (%) .
- Nested under
checks, write the checks you wish to execute against all the datasets listed under
for each dataset T: datasets: # include the dataset - dim_customers # include all datasets matching the wildcard expression - dim_products% # (optional) explicitly add the word include to make the list more readable - include dim_employee # exclude a specific dataset - exclude fact_survey_response # exclude any datasets matching the wildcard expression - exclude prospective_% checks: - row_count > 0
- Soda Core dataset names matching is case insensitive.
- If any of your checks specify column names as arguments, make sure the column exists in all datasets listed under the
- To add multiple for each configurations in your checks YAML file, configure another
for eachsection header with a different letter identifier, such as
for each dataset R.
- Need help? Join the Soda community on Slack.
Last modified on 01-Jul-22
Was this documentation helpful?
Share feedback in the Soda community on Slack.
Help improve our docs!