What's new in Soda docs?

Review a changelog of additions and revisions to Soda documentation.

March 14, 2025

• Added release notes documentation for Soda Library 1.9.3.

February 27, 2025

• Added release notes documentation for Soda Agent 1.1.39.

February 26, 2025

• Added release notes documentation for Soda Library 1.9.2.

February 24, 2025

• Added release notes documentation for Soda Library 1.9.1.

February 21, 2025

• Added release notes documentation for Soda Library 1.9.0.

February 20, 2025

• Added release notes documentation for Soda Core 3.5.0.

February 14, 2025

• Added release notes documentation for Soda Library 1.8.19.

February 6, 2025

• Added release notes documentation for Soda Agent 1.1.38.

February 4, 2025

• Added release notes documentation for Soda Library 1.8.16 - 1.8.18.

January 28, 2025

• Added release notes documentation for Soda Library 1.8.14.

January 9, 2025

• Added release notes documentation for Soda Library 1.8.13.

January 8, 2025

• Added release notes documentation for Soda Library 1.8.12.

January 7, 2025

• Added release notes documentation for Soda Library 1.8.10 - 1.8.11 and Soda Core 3.4.4.

January 6, 2025

• Added release notes documentation for Soda Library 1.8.6 - 1.8.9.

November 29, 2024

• Published a new How To guide for building a custom Soda data quality dashboard in Grafana.

November 28, 2024

• Added release notes documentation for Soda Core 3.4.2.

November 27, 2024

• Added release notes documentation for Soda Library 1.8.4.

November 26, 2024

• Added release notes documentation for Soda Library 1.8.3.

November 19, 2024

• Added release notes documentation for Soda Agent 1.1.34.

November 15, 2024

• Added instructions for optimizing Soda Agent performance using Change sample data and failed rows memory limit
• Added release notes documentation for Soda Agent 1.1.33.

November 14, 2024

• Added release notes documentation for Soda Library 1.8.2.

November 13, 2024

• Added release notes documentation for Soda Library 1.8.1.

November 1, 2024

• Updated anomaly detection check documentation to include support for freshness and user-defined metrics.

October 29, 2024

• Added release notes documentation for Soda Library 1.8.0 and Soda Agent 1.1.32.

October 28, 2024

• Add to Soda Agent extras with instructions on how to use an existing Kubernetes secret for Soda Cloud API keys.

October 25, 2024

• Added release notes documentation for Soda Library 1.7.1.

October 23, 2024

• Added instructions for installing pydanticv1 extra library package; see Troubleshoot section of installation instructions.

October 22, 2024

• Added release notes documentation for Soda Core 3.4.1.

October 21, 2024

• Published a new How To guide for building a custom Soda data quality reporting dashboard in Sigma.
• Added release notes documentation for Soda Core 3.4.0.

October 17, 2024

• Added release notes documentation for Soda Library 1.6.5 and 1.7.0, and Soda Cloud Bulk-edit of dataset responsibilities and attributes.

October 16, 2024

• Added release notes documentation for Soda Agent 1.1.31.

October 10, 2024

• Added an optional auto_exclude_anomalies parameter for anomaly detection that you can use to ignore or include existing anomalies in a training dataset.

October 9, 2024

• Added release notes documentation for Soda Agent 1.1.30.
• Updated Test data quality in a Databricks pipeline to include, among minor edits, Input data checks and model output checks.

October 8, 2024

• Added release notes documentation for Soda Library 1.6.4.

October 7, 2024

• Updated the Soda Library Python API reference documentation with attributes for samples limit, and an optional config for using variables.

October 3, 2024

• Added documentation for using variables in the SampleRef message parameter for Python custom sampler to collect and display failed row samples.

October 2, 2024

• Added documentation of new parameters for the soda-pandas-dask package to address changed behavior when upgrading to version 1.6.4 or greater. See Add optional parameter for COUNT and Add optional parameter for text data conversion.
• Added example for SampleRef with Python Custom Sampler to direct users to a bespoke location to find failed row samples for checks with failed test results.
• Added details for assigning global roles to users or user groups.

September 30, 2024

• Added Soda Library Python API reference documentation.

September 26, 2024

• Updated documentation to include customizable permissions for global and dataset roles in Soda Cloud, plus the ability to create new roles.
• Added release notes documentation for Soda Library 1.6.3 and Soda Agent 1.1.29.

September 25, 2024

• Added release notes documentation for Soda Agent 1.1.28.
• Published Soda product release states to describe the status of newly-released features or functionality.

September 24, 2024

• Add Configuration and setting hierarchy section to offer an overview of behavior for failed row sample collection.
• Added release notes documentation for Soda Library 1.6.2.
• Removed note about an upper limit of 10,000 for collecting failed row samples.
• Added Soda Cloud connection configuration details to Connect to Dask and Pandas, and corrcted install package names in Troubleshooting section.
• Added troubleshooting solution to MS Teams integration.

September 23, 2024

• Compiled and updated failed row samples documentation, including:

  • the option to use scan context in a CustomSampler to read/write data to/from a scan
  • the option to collect failed rows samples from specific columns in a dataset in Soda Cloud
  • the option to disable failed row sample collection from all datasets, except those with explicit configuration to collect samples
• Updated Failed row checks and User-defined checks to include optional configuration to specify a single column against which to run the check.
• Revised check attributes configuration when applying attributes to more than one check.

September 23, 2024

• Moved data contract lanugage reference content to soda-core GitHub repository to avoid confusion with SodaCL reference.

September 19, 2024

• Added attribute mapping to Okta SSO integration documentation.
• Correct reconciliation check documentation to remove the option to add a list of comma-separated datasets to compare.

September 18, 2024

• Added release notes documentation for Soda Agent 1.1.27.

September 17, 2024

• Added release notes documentation for Soda Library 1.6.1.

September 13, 2024

• Added egress IP addresses for Soda Cloud. See: Receiving events from Soda Cloud.

September 12, 2024

• Added release notes documentation for Soda Core 3.3.15 - 3.3.22.

September 8, 2024

• Published new use case guide for using Soda to test data quality in a Dasger pipeline.

September 4, 2024

• Added release notes documentation for Soda Library 1.6.0 and Soda Agent 1.1.26.

August 29, 2024

• Published new use case guide for using Soda to test data quality in an Azure Data Factory pipeline.

August 28, 2024

• Updated documentation to clarify when to deploy a self-hosted vs. Soda-hosted agent.

August 20, 2024

• Published new use case guide for using Soda to test data quality in a Databricks pipeline.

August 19, 2024

• Added release notes documentation for Soda Agent 1.1.25.

August 14, 2024

• Added release notes documentation for Soda Library 1.5.25 and Soda Core 3.3.14.

August 13, 2024

• Added release notes documentation for Soda Library 1.5.24 and Soda Agent 1.1.24.

August 8, 2024

• Added content to clarify that Soda Library officially supports Python 3.8, 3.9, and 3.10.

August 2, 2024

• Added release notes documentation for Soda Library 1.5.23 and Soda Agent 1.1.23.

August 1, 2024

• Added release notes documentation for Soda Library 1.5.22 and Soda Agent 1.1.22.

July 31, 2024

• Added release notes documentation for Soda Library 1.5.21.
• Added troubleshooting tip for running Soda scans on Databricks where column names beging with numbers.
• Added Known issues and limitations section to anomaly dashboard content.

July 29, 2024

• Added release notes documentation for Soda Core 3.3.13.

July 25, 2024

• Documented instructions for how to add one-way user group synchronization from an SSO IdP to Soda Cloud.
• Updated data type support for MS SQL Server to include NCHAR, NVARCHAR, and BINARY.

July 24, 2024

• Added release notes documentation for Soda Library 1.5.20 and Soda Agent 1.1.21.

July 23, 2024

• Added release notes documentation for Soda Library 1.5.19.

July 22, 2024

• Added release notes documentation for Soda Library 1.5.18.
• Updated compatibility for anomaly dashboard preview activation.
• Added clarification for Soda compatibility with IBM DB2 data sources, LUW vs. z/OS.

July 19, 2024

• Updated MS Teams integration documentation to reference creating Workflows in MS Teams instead of Office 365 Connectors. Microsoft is retiring the connectors effective August 15, 2024. If you have previously set up a Soda integration with an Office 365 connector, follow the instructions for Creating a workflow from a channel in Teams, then update the integration URL in your existing Soda <> MS Teams integration in Soda Cloud.

July 18, 2024

• Added release notes documentation for Soda Agnet 1.1.20 and Soda Core 3.3.12.
• Added host and port as optional connection configuration parameters for Snowflake.
• Added an optional multi_subnet_failover parameter to the connection configuration for MS SQL.
• Added an optional sslmode parameter to the connection configurations for PostgreSQL and Denodo.

July 17, 2024

• The preview program for anomaly dashboards for observability has reached its quota. Removed “Request preview access” links from documentation.
• Added release notes documentation for Soda Library 1.5.17 and Soda Core 3.3.11.
• Corrected Missing metrics and Validity metrics to indicate that column config keys that use regex are supported only for text data types.

July 16, 2024

• Added release notes documentation for Soda Library 1.5.16 and Soda Agent 1.1.19.

July 15, 2024

• Added release notes documentation for Soda Library 1.5.15.

July 10, 2024

• Revised SodaGPT documentation to replace it with details about Ask AI, Soda’s in-product generative AI assistant.

July 8, 2024

• Documented the new functionality that enables Admin users in Soda Cloud to create user groups.
• Added release notes documentation for Soda Core 3.3.10.

July 5, 2024

• Added clarification to the inclusion and exclusion rules for profiling behavior.
• Repeated the configuration instructions for samples columns when implicitly collecting failed row samples in multiple places, notably in Collect failed row samples.
• Added details about RollingUpdate when upgrading a self-hosted Soda Agent.

July 2, 2024

• Added release notes documentation for Soda Agent 1.1.17 & 1.1.18 and Soda Library 1.5.14.

June 28, 2024

• Added release notes documentation for Soda Agent 1.1.15 & 1.1.16, Soda Library 1.5.13, and Soda Core 3.3.7, 3.3.8 & 3.3.9.
• Published documentation to accompany data contracts version 4 release.

June 27, 2024

• Added release notes documentation for Soda Agent 1.1.14 and Soda Library 1.5.12.

June 24, 2024

• Added release notes documentation for Soda Agent 1.1.13 and Soda Library 1.5.11.

June 21, 2024

• Added release notes documentation for Soda Agent 1.1.12 and Soda Library 1.5.10.
• Documented how to double-onboard a data source.

June 10, 2024

• Added release notes documentation for Soda Core 3.3.6 and Soda Library 1.5.9.

June 18, 2024

• Added release notes documentation for Soda Agent 1.1.10 & 1.1.11 and Soda Library 1.5.7 & 1.5.8.

June 17, 2024

• Added a new docs page to begin recording data source connection issues and workarounds.
• Added link to troubleshooting advice for reference checks that use dataset filters.
• Added troubleshooting advice for Snowflake connections that use proxies.
• Clarified the use of scan definition names in multiple programmatic scans in different pipelines.
• Added requirement for SSO setup for customers to indicate whether they use Identity Provider Initiated (IdP-initiated) or and Service Provider Initiated (SP-initiated) single sign-on integrations. (Also included in procedural instructions.)
• Updated experimental support for data contracts.

June 10, 2024

• Added release notes documentation for Soda Agent 1.1.9 and Soda Library 1.5.6.

June 7, 2024

• Added release notes documentation for Soda Agent 1.1.8.

June 6, 2024

• Added release notes documentation for Soda Agent 1.1.7.

June 5, 2024

• Added release notes documentation for Soda Library 1.5.5.
• Added details about IRSA authentication for Athena and Redshift data sources.
• Added new example script to fetch dataset and check info from a Soda Cloud account and transfer data into CSV files.

May 30, 2024

• Added release notes documentation for the Soda AI features generally available or available for preview access upon request.

May 29, 2024

• Added release notes documentation for Soda Library 1.5.4 and Soda Agents 1.1.5 and 1.1.6.
• Added documentation and example of including a failed rows query in a user-defined check.

May 28, 2024

• Added release notes documentation for Soda Library 1.5.3.

May 25, 2024

• Added release notes documentation for Soda Agent 1.1.4.

May 24, 2024

• Added release notes documentation for Soda Library 1.5.2, Soda Core 3.3.5, and Soda Agent 1.1.3.

May 23, 2024

• Documented the new feature for data quality observability, the automated, ML-driven Anomaly Dashboard.
• Added release notes documentation for Soda Library 1.5.1 and Soda Agent 1.1.2.
• Added note about using a different key for database for connecting to a DuckDB data source.

May 20, 2024

• Added release notes documentation for Soda Library 1.5.0.

May 17, 2024

• Added release notes documentation for Soda Library 1.4.10.

May 14, 2024

• Added release notes documentation for Soda Agent 1.1.1 and Soda Library 1.4.9.

May 8, 2024

• Added to programmatic scan to include option to run the scan locally and not send results to Soda Cloud.

May 7, 2024

• Added to example script for a programmatic scan to include a check template file path.
• Added prerequisite to no-code check creation that datasets must be discovered during data source onboarding.
• Touched up some details for advanced configuration of the anomaly detection simulator.
• Added release notes documentation for Soda Core 3.3.3 and 3.3.4, and Soda Library 1.4.8.

May 6, 2024

• Removed the Agreement deprecation notice as the decision to deprecate the feature has been reversed.

April 30, 2024

• Published documentation for V3 of data contracts, Soda’s experimental way to set data quality standards for data products.

April 29, 2024

• Added optional connection parameters to Denodo data source configuration.

April 26, 2024

• Included information about allocating resources for improved performance of a self-hosted Soda Agent.

April 25, 2024

• Added release notes documentation for Soda Agent 1.1.0.
• Added documentation to initiate an integration between Soda Cloud and Microscoft Purview.

April 24, 2024

• Added release notes documentation for Soda Core 3.3.2.

April 22, 2024

• Added release notes documentation for Soda Agent 1.0.8-10.

April 12, 2024

• Added infomation about using Soda and Airflow.

April 10, 2024

• Added release notes documentation for Soda Library 1.4.7, and Soda Agent 1.0.6 and 1.0.7
• Documented new parameters for Dremio data source connections.

April 4, 2024

• Added release notes documentation for Soda Library 1.4.5 & 1.4.6, and Soda Agent 1.0.4 and 1.0.5.
• Updated the example script in Reroute failed row samples guide.

March 27, 2024

• Added release notes documentation for Soda Core 3.3.1, Soda Library 1.4.4, and Soda Agent 1.0.3.

March 21, 2024

• Added release notes documentation for Soda Library 1.4.3.
• Added more content and example for rerouting failed row samples.
• Added requirement for using anomaly detection checks in group by configurations: requires Soda Library 1.1.27 or greater, or Soda Agent 0.8.57 or greater.

March 20, 2024

• Updated Oracle connection configuration to use a more generic connectstring value.
• Prepred a separate reference section for data contract checks.

March 18, 2024

• Added release notes documentation for Soda Core 3.2.4 and 3.3.0.
• Added details for passlisting domain names for Soda Agent to communicate with Soda Cloud.

March 15, 2024

• Published documentation for V2 of data contracts, Soda’s experimental way to set data quality standards for data products.

March 12, 2024

• Add instructions for how to programmatically use Soda Library with an example script to reroute failed row samples to the CLI output instead of Soda Cloud.

March 6, 2024

• Update Soda integration with dbtCloud to include instruction for dbt’s new access_URL.
• Updated the Integrate Soda with Microsoft Teams documentation to accommodate new MS isntructions for creating an incoming webhook.

March 5, 2024

• Added release notes documentation for Soda Agent 1.0.1.
• Added MS SQL Server and Redshift to the list of data sources you can connect to using a Soda-hosted Agent.
• Added details to Integrate with Alation documentation to access an Alation account guarded by SSO.
• Added example for loading a JSON file into a Dataframe using Dask and Pandas.
• Added example for comparing partitioned datasets in different schemas in the same data source.
• Added release notes documentation for Soda Library 1.4.2, Soda Core 3.2.3, Soda Agent 1.0.2, and Soda Agent 0.9.2.

March 1, 2024

• Published guidance for managing sensitive data in Soda.
• Added release notes documentation for Soda Library 1.4.1.
• Added a compatibility legend to SodaCL reference documentation to clarify which checks are available via various means; see example.

February 29, 2024

• Following improvements and changes to the self-hosted Soda Agent 1.0.0, removed the documented details for including idle replicas and polling intervals in a cluster that aimed to improve scan times. Also, added release notes to inform existing Soda Agent users about changes to parameter configuration with 1.0.0 and advice for optimal performance using managed node groups instead of Fargate profiles in Amazon EKS, GCP Autopilot, or AKS Virtual Clusters. See Soda Agent release notes for upgrade details.
• Added details for system requirements for deploying a Soda Agent in a Kubernetes cluster.
• Included schema checks as available to add as a no-code check to a dataset in a data source that uses a Soda Agent to execute scans.
• Added instructions for how to run a Soda Cloud-defined scan remotely using the Soda Library CLI. See the Remotely run a scan tab in Scan for data quality.
• Added release notes documentation for Soda Library 1.3.4, 1.4.0 and Soda Core 1.3.3.
• Added Databricks SQL to the list of data sources you can connect to using a Soda-hosted Agent.

February 28, 2024

• Added notation for opting out of usage statistics with a Soda Agent.
• Added notation that Group By checks support a maximum of 1000 groups.
• Changed instances of scheduled scan and scan schedule to scan definition to match the Soda Cloud user interface.
• Clarified the support for casting columns when using a freshness check.
• Clarified the Basic SAML Configuration values to provide during SSO integration with Azure AD.

February 26, 2024

• Added release notes documentation for Soda Agent 0.9.1, which maps to Soda Library 1.3.2.

February 22, 2024

• Updated Soda Cloud API documentation to clarify details.

February 21, 2024

• Added API documentation for the Soda Cloud API that enables you to trigger Soda Cloud scans programmatically.
• Added a new section to Scan for data quality for triggering a scan via API.
• Added release notes documentation for Soda Agent 0.9.0, which maps to Soda Library 1.3.2.

February 14, 2024

• Updated connection configuration parameters for Athena and Oracle.
• Made corrections to the connection details for Athena and Redshift; access keys are required parameters for each, regardless of whether you also use a role_arn parameter.

February 13, 2024

• Added release notes documentation for Soda Library 1.3.3.
• Added release notes documentation for Soda Library 1.3.2 and Soda Core 3.2.1.
• Added release notes documentation for Soda Agent 0.8.57, which maps to Soda Library 1.3.2.

February 9, 2024

• Published documentation for the new ability to add multiple group by configurations, and instructions for how to preserve historical measurements when making changes to a group by configuration.
• Added release notes documentation for Soda Library 1.3.1.

February 8, 2024

• Documented how to use the anomaly detection simulator.
• Added release notes documentation for Soda Library 1.3.0 and Soda Core 3.2.0.

February 2, 2024

• Added links to a video that demonstrates how to add Soda to a Databricks pipeline.
• Added release notes documentation for Soda Agent 0.8.56, which maps to Soda Library 1.2.4.

February 1, 2024

• Published new documentation for the Soda-hosted agent, a secure, out-of-the-box agent you can use to connect to data sources from within the Soda Cloud user interface.
• Added documentation for the new anomaly detection check, which replaces the anomaly score check.
• Added release notes documentation for Soda Agent 0.8.55, which maps to Soda Library 1.2.3.

January 29, 2024

• Added release notes documentation for Soda Agent 0.8.54, which maps to Soda Library 1.2.3.

January 26, 2024

• Added release notes documentation for Soda Library 1.2.2 & 1.2.3 and Soda Core 3.1.4 & 3.1.5.
• Added instructions for customizing a dashboard.

January 22, 2024

• Updated the documentation for rerouting failed row samples to include new, optional configuration parameters that offer users direct access to the failed row sample data.

January 19, 2024

• Updated compatible data sources for Soda Agent to include Databricks SQL.

January 15, 2024

• Added release notes documentation for Soda Library 1.2.1.

January 12, 2024

• Documented configuration changes and performance improvements for record reconciliation checks.
• Added release notes documentation for Soda Library 1.2.0.

January 5, 2024

• Published a new use case guide for integrating an External Secrets Manager with a Soda Agent.
• Adjusted Roles and Rights in Soda Cloud to accommodate licensing models that are not based on Author or Viewer volumnes.

January 3, 2024

• Updated Integrate Jira with Soda to include copy-able code snippets for the field values in Jira.
• Documented the optional syntax for anomaly score checks to produce warnings instead of fails.
• Added release notes documentation for Soda Library 1.1.29 and Soda Core 3.1.3.

January 2, 2024

• Added alertnate syntax for failed row check using a failed row condition.

December 21, 2021

• Documented the support for tracking anomalies and changes over time in checks grouped by category.
• Updated the Self-serve Soda use case guide to include instructions for using no-code checks and Discussions to empower non-coders to join the team effort of establishing good-quality data.

December 15, 2023

• Added release notes documentation for Soda Library 1.1.27 and Soda Agent 0.8.53.
• Added release notes documentation for Soda Library 1.1.28 and Soda Core 3.1.2.

December 13, 2023

• Updated freshness check to include support for in-check filters.
• Added documentation to clarify that Soda supports Azure Data Factory (ADF) with Airflow using Synapse connection configuration.
• Documented the support for adding quotes to all datasets that Soda acts upon automatically such as with profiling or discovering datasets.
• Added an example of an in-check filter that uses a string value.
• Added a troubleshooting item for the error NoneType object is not iteratable.
• Added instructions for dynamically including a dataset name in a for each configuration.
• Prepared new, independent documentation for integrating Soda with Jira and ServiceNow.

December 7, 2023

• Introducting no-code check creation in Soda Cloud. Create checks via the Soda Cloud user interface that creates SodaCL checks without writing any SodaCL.

December 4, 2023

• Added release notes documentation for Soda Library 1.1.26, Soda Core 3.1.1, and Soda Agent 0.8.51 - 0.8.52.

November 29, 2023

• Corrected the example included in User-defined checks.

November 28, 2023

• Added to Best practice for using reconciliation checks with advice on batch processing.
• Added instruction for using reference checks with DataFrames; see Use Soda Library with Spark DataFrames on Databricks.
• Added content to the Self-serve Soda use case guide with a list of compatible data sources and a link to data source configuration reference content.

November 24, 2023

• Added release notes documentation for Soda Library 1.1.24 - 1.1.25.
• Added Known issue to Group By configuration; does not support anomaly score checks.
• Adjusted workaround advice for troubleshooting error using quotes with an in-check filter.
• Added Advanced configuration for setting key column identifiers.

November 22, 2023

• Added an example of a schema check that detects columns which could contain PII.
• Added release notes documentation for Soda Agent 0.8.49 - 0.8.50.

November 21, 2023

• Added documentation for managing scans and setting up failed scan notifications.
• Added work_group as an optional connection configuration property for Athena.
• Added troubleshooting tip for using quotes on column names within an in-check filter. See Troubleshoot SodaCL.
• In the context of Soda Cloud, changed instances of scan defintion to scan schedule to reflect the updated naming in the Soda Cloud UI.

November 16, 2023

• Introducing the launch of data contracts, Soda’s experimental way to set data quality standards for data products.
• Added release notes documentation for Soda Core 3.1.0.

November 15, 2023

• Corrected the rule that numeric characters in a list of valid values, invalid values, or missing values, must be wrapped in single quotes. This is not the case. See Specify valid or invalid values for corrected content.

November 14, 2023

• Added release notes documentation for Soda Library 1.1.22 and Soda Core 3.0.54.

November 8, 2023

• Removed Reporting API v0 documentation as the version is now deprecated.

November 7, 2023

• Added two configuration keys for use with validity metrics: invalid format, invalid regex.
• Soda Cloud Reporting API v0 is now deprecated. Please use Reporting API v1.
• Updated data source configuration reference content to fill in blanks and offer more examples.

November 2, 2023

• Added pollingInterval to Soda Agent deployment instructions.
• Added release notes documentation for Soda Library 1.1.20 - 1.1.21 and Soda Core 3.0.52 - 3.0.53.
• Added sample input values and clarifying notes to data source connection config reference for Athena.

October 30, 2023

• Updated anomaly score documentation to include support for dataset filters.
• Added documentation to accompany new support for Presto data source.

October 26, 2023

• Added documentation to accompany new support for MotherDuck data source.

October 25, 2023

• Added to the list of supported check types in SodaGPT.
• Added another example snippet to Group By checks.

October 24, 2023

• Added instructions to Connect Soda to Spark to recommend changing the name of the data_source_name in step 5.

October 23, 2023

• Added release notes documentation for Soda Library 1.1.19.
• Clarify instructions about adding a check identity to a check; see Add a check identity.
• Corrected the syntax for data source connection values when using the GitHub Action for Soda in a Workflow; needed spaces before and after variables in single curly braces. See Add the GitHub Action for Soda to a Workflow.
• Added Slack icon in header to link to Soda Community.

October 17, 2023

• Deprecated sampling from distribution check DRO generation.
• Documented the support for adding alert coniditions to a failed row check.
• Added instructions for applying check attributes to multiple checks in a single checks for dataset_name block.

October 13, 2023

• Added new content to clarify what an active check is. Soda’s licensing model can inlcude volume-based measures of active checks.
• Added link to new video for Atlan integration.

October 12, 2023

• Added release notes documentation for Soda Library 1.1.17 - 1.1.18.
• Removed support for quotes in dataset name identifiers in checks; see Use quotes in a check.
• Adjusted instructions for Connect Soda using Dask and Pandas.

October 11, 2023

• Refactored the content on docs.soda.io to focus more on use cases, tasks, and reader goals. The goal of the project was to pivot from a products-based set of documentation to task-based/use case-based content. You may notice a change to the navigation on docs.soda.io that is organized by actions (Install, Deploy, Run Scans, Set alerts, etc.) instead of by product (Soda Library, Soda Cloud, Soda Agent, etc.)

  • Access a new Get started roadmap with recommendations to help you quickly become productive and confident using Soda for data quality testing.
  • Get inspired by new Use case guides to offer guidance in setting up Soda to meet a specific need.
  • Get your Soda account organized and set up to maximize your team’s data quality testing efficiency.
• Updated Integrate Soda with dbt to install sub-packages with double-quotes.
• Update best practices for reconciliation checks to recommend creating a separate agreement for a reconciliation project.
• Added release notes documentation for Soda Library 1.1.15 - 1.1.16 and Soda Core 3.0.51.

October 6, 2023

• Updated session_parameters config to session_params in Snowflake connection config reference.
• Added instructions for how to reset anomaly history for an anolamy score check.
• Added detail to programmatic scan to include a filename in a scan when checks are included inline.
• Added release notes documentation for Soda Library 1.1.14.

October 5, 2023

• Added release notes documentation for Soda Cloud dashboard.
• Added schema_name parameter to DuckDB configuration.

September 27, 2023

• Added clarifying information about user input and how it is used by SodaGPT.
• Added release notes documentation for Soda Library 1.1.13 and Soda Core 3.0.50.

September 26, 2023

• Added documentation for reconciliation schema checks which now support data type mapping.
• Documented a new scan option, --local that you can add to a soda scan command to prevent Soda Library from pushing any check results to Soda Cloud. See: Add scan options and Scan output in Soda Cloud.
• Revised and tigtened Soda Core information.
• Documented the global configuration to disable sending any samples of data to Soda Cloud; see Disable samples in Soda Cloud.

September 21, 2023

• Updated support for dbt for ingesting tests into Soda Cloud. You must now install a soda-dbt subpackage that uses dbt 1.5 or 1.6.
• Added release notes documentation for Soda Library 1.1.12.

September 20, 2023

• Updated schema reconciliation checks to clarify that the check validates columns names and data types.

September 19, 2023

• Added release notes documentation for Soda Library 1.1.11 and Soda Core 3.0.49.

September 18, 2023

• Added to Reference check documentation for the new configuration must not exist.
• Updated support for dbt-core 1.3, 1.5, and 1.6 for ingesting tests into Soda Cloud.
• Added documentation for schema reconciliation checks.

September 14, 2023

• Removed known issue for inability to use check identity with failed row checks. This is now supported in Soda Library.

September 13, 2023

• Added release notes documentation for Soda Library 1.1.10.

September 12, 2023

• Added release notes documentation for Soda Library 1.1.9.

September 11, 2023

• Added release notes documentation for Soda Library 1.1.6 - 1.1.8.
• Added a new section for Best practice for using reconciliation checks

September 1, 2023

• Added release notes documentation for Soda Library 1.1.0 - 1.1.5.
• Added item to Troubleshoot section for Snowflake.

August 31, 2023

• Added documentation for SodaCL reconciliation checks, tailored for data migration use cases.
• Added release notes documentation for Soda Library 1.1.0 - 1.1.5.

August 30, 2023

• Added instructions for integrating with an external secrets manager with a Soda Agent to manage frequently-changed data source login credentials.
• Added screenchots to Integrate Soda with Atlan documentation.
• Added to Troubleshoot content for running a scan that produces an SSL certificate error.

August 24, 2023

• Added documentation for the new, native integration of Soda in Atlan.
• Updated orchestration documentation to include a link to an Astronomer tutorial for Data Quality Checks with Airflow, Snowflake, and Soda.
• Added to item to Troublshoot SodaCL for dealing with unexpected missing checks behaviour.

August 23, 2023

• Update agreement documentation to reflect the change in behaviour where scans do not run until stakeholders have approved of the agreement.

August 21, 2023

• Removed “What the Action does” section from Integrate Soda with a GitHub Workflow.

August 11, 2023

• Added release notes documentation for Soda Core 3.0.48.
• Added release notes documentation for Soda Library 1.0.6 - 1.0.8.
• Added Known issue: Failed rows checks do not support the check identity parameter.
• Added a note to Create an agreement to clarify that you can only create agreements using data sources that have been added to Soda Cloud via a Soda Agent.
• Added collection as a new term in the Glossary.

August 10, 2023

• Published new documentation for the GitHub Action for Soda.
• Updated Test data during development to replace the GitHub Action recipe with the new GitHub Action for Soda.

August 8, 2023

• Revised documentation to reflect the new Checks dashboard, that displays checks and their latest scan results. This replaces the Check Results dashboard, that displayed all individual check results.

August 7, 2023

• Moved Check suggestions documentation from SodaCL section to Soda Library.

July 26, 2023

• Added release notes documentation for Soda Library 1.0.5.
• Added detail to schema check documentation for new schema_name parameter.

July 24, 2023

• Added support for failed row checks when using check templates.

July 21, 2023

• Added documentation to complement Google CloudSQL support.
• Added release notes documentation for Soda Library 1.0.3 and Soda Library 1.0.4.
• Added release notes documentation for Soda Core 3.0.42 - 3.0.47.
• Added new http_headers configuration parameter for Trino data source.

July 6, 2023

• Added documentation for the samples columns check configuration for metrics and checks that implicitly collect failed row samples: missing, validity, duplicates, reference.

July 4, 2023

• Updated commands for installing Soda Library using a Docker image.

June 27, 2023

• Documentation to accompany the preview launch of SodaGPT.

June 23, 2023

• Changed requirement for check template to include the dataset identifier in the first line of the check so that Soda Cloud can properly render the check results.
• Added release notes documentation for Soda Core 3.0.40 and Soda Core 3.0.41.
• Added release notes documentation for Soda Library 1.0.1 and Soda Library 1.0.2.
• Reverted Soda Agent to describe configuring Soda Core settings instead of Library. Will update to Soda Library details when updates are complete.

June 15, 2023

• Introducing Soda Library, a commercial extension of the Soda Core open-source software. It leverages all the power of Soda Core and SodaCL, and offers new features and functionality for Soda customers.
• New documentation for the new Group by configuration and Group evolution check, both available with Soda Library.
• New documentation for Check suggestions using the Soda Library CLI.
• New documentation for Check template configuration supported by Soda Library.
• Revised syntax guidance regarding multiple thresholds for an alert. See Optional check configurations.
• All documentation for Soda Core, the open-source Python library and CLI tool, has moved to the Soda Core repository on GitHub.

June 12, 2023

• Remove “Preview” tag from the Reporting API v1 documentation.

June 9, 2023

• Added Known Issue for using BigQuery and specifying numeric missing values or valid values with single quotes. TL;DR: Don’t use single quotes.
• Added clarification to the value for path when connecting a DuckDB data source.
• Removed incorrect syntax guidance regarding multiple thresholds for an alert. Each warn or fail condition can contain only one threshold. See Optional check configurations.
• Updated instructions for configuring a soda_cloud connection in a configuration.yml file. New instructions involve copying the whole configuration instead of just API Key values.

June 8, 2023

• Added release notes documentation for Soda Core 3.0.38 and Soda Core 3.0.39.

May 31, 2023

• Added instructions and event payload details for using a webhook to notify a third-party of new, deleted, or changed Soda agreements.

May 30, 2023

• Added a new parameter, datasource_container_id to the .datasource-mapping.yml file neede to map a Soda Cloud-Alation catalog integration.

May 25, 2023

• Added a step for configuring soda-core-spark[databricks] to be sure to install databricks-sql-connector as well.

May 23, 2023

• Added a video overview showcasing the integration of Soda and Alation.
• Added a note for a Known Issue regarding the use of variables in profiling configurations.

May 20, 2023

• Added documentation for using private key authentication for Snowflake when deploying a Soda Agent.

May 15, 2023

• Replaced getting started guides with entirely new content with a focus on data engineering.

  • Take a sip of Soda
  • Test data in a pipeline
  • Test data during development
  • Enable end users to test data
• Replaced the product overview with newly-written material.

May 11, 2023

• Added release notes documentation for Soda Core 3.0.33 and Soda Core 3.0.34.
• Added instructions for user-defined metrics to access and use queries in separate SQL files.
• Adjusted content for the revised CLI and Soda Cloud scan output for schema checks. Schema check results now display the output for all alerts triggered during a scan.

May 9, 2023

• Added the install package to each connector’s page.
• Added a connectivity troubleshooting tip to Connect to Snowflake.

May 2, 2023

• Published content regarding the set up of multiple Soda Cloud organizations for use with different environments in your network infrastructure.
• Added a note about selecting a region when you sign up for a new Soda Cloud account.

April 28, 2023

• Corrected the explanation of the duplicate_count check regarding checks that included multiple arguments (columns).

April 18, 2023

• Added release notes documentation for Soda Core 3.0.31 and Soda Core 3.0.32.

April 11, 2023

• Added a copy-to-clipboard button to most code snippets in documentation.
• Added attribute mapping details to add Soda Cloud to Google Workspace as a SAML app.

March 29, 2023

• Revised instructions to add Soda Cloud to Google Workspace as a SAML app.

March 28, 2023

• Added to Soda Agent documentation to include a setting for which Soda Cloud endpoint to use, according to region. See Deploy an Soda Agent in a Kubernetes cluster.

March 24, 2023

• Added content to Troubleshoot SodaCL to address challenges when using a reference check with a dataset filter.
• Added instructions to add Soda Cloud to Google Workspace as a SAML app.
• Added parameter to Snowflake connection details for using private key encryption for private key authentication.

March 21, 2023

• Added release notes documentation for Soda Core 3.0.29 and Soda Core 3.0.30.
• Added instructions for limiting samples for an entire data source.

March 9, 2023

• Added release notes documentation for Soda Core 3.0.28.

March 8, 2023

• Added to Troubleshoot SodaCL with information about checks that return [NOT EVALUATED] results.
• Added new content with advice to Compare data using SodaCL.
• Documented how to prevent Soda from collecting failed rows samples and sending them to Soda Cloud using a samples limit.
• Corrected a prerequisite in Add a data source to indicate that you can deploy a Soda Agent in any Kubernetes cluster, not just Amazon EKS.

March 7, 2023

• Added release notes documentation for Soda Core 3.0.26 & 3.0.27.

February 28, 2023

• Published instructions for setting up private connectivity to a Soda Cloud account using AWS PrivateLink.

February 23, 2023

• Documented known issue with freshness check. See Troubleshoot errors with freshness checks.
• Added release notes documentation for Soda Core 3.0.24 & 3.0.25.

February 22, 2023