Release notes for Soda products

Review release notes for all Soda data observability products.

Join to the #release-alerts channel in Slack to get notifications of Soda product releases.[soda-library] 1.11.2

^{07 April 2025}

1.11.2 Features and Fixes

Trino: fix query pagination by @m1n0 in #490

[soda-runner] 1.1.46

^{07 April 2025}

1.1.46

Bump library to 1.11.2 by @m1n0 in #120

[soda-library] 1.11.1

^{04 April 2025}

1.11.1 Features and Fixes

New: Reconciliation Reference check by @m1n0 in #488

[soda-runner] 1.1.45

^{04 April 2025}

1.1.45

Bump library to 1.11.1 by @m1n0 in #119

[soda-library] 1.10.4

^{03 April 2025}

1.10.4 Features and Fixes

fix(dbt): handle missing column_name in test node by @adkinsty in #486

[soda-library] 1.10.3

^{03 April 2025}

1.10.3 Features and Fixes

Skip databricks history until table creation by @mivds in #482
Fix backfilling when there is no partition column by @mivds in #484
Fix incorrect row count for REPLACE TABLE by @mivds in #483
Fix 1-interval offset for table creation history by @mivds in #485
Use partition column suggestions from soda cloud by @mivds in #481
Limit partition_row_count metric to data range by @mivds in #487
Set numba log level to WARNING by @mivds in #489

[soda-agent] 1.1.44

^{03 April 2025}

1.1.44

Bump soda-library to v1.10.3 by @mivds in #118

[soda-library] 1.10.2

^{01 April 2025}

1.10.2 Features and Fixes

dbt: support ingestion of dbt test results from all versions of dbt by @adkinsty in #474

[soda-agent] 1.1.43

^{01 April 2025}

1.1.43

Bump library to 1.10.2 by @m1n0 in #117

[soda-library] 1.10.1

^{31 March 2025}

1.10.1 Features and Fixes

Relax numpy requirement to < 3.0.0 by @adkinsty in #453
Oracle: add schema to dataset prefix, support custom prefix by @m1n0 in #460
Redshift: add keepalive by @m1n0 in #480
Group by: apply global attributes by @m1n0 in #475

Metric Monitoring

Enable checking for column addition by @mivds in #462
Report metadata metrics using query time by @mivds in #461
Fix schema filter in snowflake metadata query by @mivds in #463
Fall back to default userConfig if not available by @mivds in #465
Update metric monitoring check names by @mivds in #467
Limit generation of userConfig missing warning by @mivds in #466
Inform user when metadata is not available by @mivds in #470
Fix backfilling scan with no data in date range by @mivds in #469
Fix warning when there are no matching tables by @mivds in #471
Fix backfilling missing partition row count by @mivds in #472
Fix backfilling for ‘most recent timestamp’ metric by @mivds in #476
Rename partition row count by @mivds in #479
Fix handling of partition column as datetime by @mivds in #478
Alert on lower threshold for time metrics by @mivds in #477
updating algorithm’s use of feedback by @nikosml in #468
Add rounding and more algorithm testing by @nikosml in #464

[soda-library] 1.10.0

^{25 March 2025}

1.10.0 Features and Fixes

Enable metric monitoring using new algorithm by @mivds in #455
Remove support for python 3.8 by @mivds in #458

[soda-agent] 1.1.42

^{25 March 2025}

1.1.42

Bump soda-library and soda-scan-launcher versions by @mivds in #116

[soda-agent] 1.1.41

^{25 March 2025}

1.1.41

Bump soda-library and soda-scan-launcher versions by @mivds in #116

[soda-library] 1.9.3

^{14 March 2025}

1.9.3 Features and Fixes

Fix scan time reporting by @mivds in #428
Validity check: fix complex mixed rules query (#2219) by @m1n0 in #440
fix(trino): handle period character in schema name by @adkinsty in #444

[soda-agent] 1.1.40

^{05 March 2025}

1.1.40

Bump soda-scan-launcher & orchestrator versions by @Antoninj in #115

[soda-agent] 1.1.39

^{27 February 2025}

1.1.39

Releases Soda Library 1.8.19.

Improve the documentation a bit by @milanaleksic in #111
Add support for soda-contract-launcher (a.k.a. v4) by @nielsn in #112
Bump soda-library and soda-scan-launcher versions by @mivds in #114

[soda-library] 1.9.2

^{26 February 2025}

1.9.2 Features and Fixes

Postgres: treat missing setting as warning by @mivds in #403
Use calendar chart for schema check by @mivds in #401
Fix reporting profiling for distributed scans by @mivds in #407

[soda-library] 1.9.1

^{24 February 2025}

1.9.1 Features and Fixes

Extend observability metadata support by @mivds in #399
Remove deprecated anomaly detection diagnostics by @mivds in #395
Fix date formatting for years before 1000 by @mivds in #397

[soda-library] 1.9.0

^{21 February 2025}

1.9.0 Features and Fixes

Trino: Add CI / CD, fix tests by @dakue-soda in #384
Profiling: Skip profiling if disabled on Cloud by @m1n0 in #389
Metric Monitoring: Wip MM with backfilling by @m1n0 in #387
Athena: Add session token parameter by @m1n0 in #391
Postgres: Handle unsupported options parameter by @m1n0 in #392
Chore: Remove unused MarkupSafe dependency by @m1n0 in #393
Chore: Relax opentelemetry version contraint by @m1n0 in #394

[soda-core] 3.5.0

^{20 February 2025}

3.5.0 Features and fixes

Custom identity: Support in Reference and Group Evolution by @m1n0 in #2208
Validity: Accept 24-hr times in ISO 8601 dates by @pholser in #2133
Dask: Fix wrong count query when row_count used with duplicate check on two columns. by @jzalucki in #2209
Dask: Fix COUNT queries on windows by @mivds in #2210
Impala: Add support for Apache Impala by @marcocharlie in #2191
Athena: Add session token parameter by @ruixuantan in #2095
Postgres: Handle unsupported options parameter by @RekunDzmitry in #2176
Duckdb: Relax duckdb dependency to < 1.1.0 by @tombaeyens in #2162
Docs: Fix typo by @crazyshrut in #2151
Docs: Fix workflow path for CI by @syou6162 in #2206
Chore: Remove unused MarkupSafe dependency (#2103) by @ghjklw in #2200
Chore: Relax opentelemetry version contraint (#2192) by @ghjklw in #2199

Refer to the Soda Core Release Notes for details.

[soda-library] 1.8.19

^{14 February 2025}

1.8.19 Features and Fixes

Fix dask COUNT queries on windows by @mivds in #385
DT-158 - add JWT authentication to Trino data sources by @dakue-soda in #386

[soda-library] 1.8.18

^{04 February 2025}

1.8.18 Features and Fixes

Fix in dask wrong count query when row_count used with duplicate check on two columns. by @jzalucki in #381
Custom identity: support in Reference and Group Evolution (#2208) by @m1n0 in #382
ISO 8601 date: accept 24-hr times (#2133) by @m1n0 in #383

[soda-agent] 1.1.38

^{04 February 2025}

1.1.38

Releases Soda Library 1.8.18.

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.8.17

^{30 January 2025}

1.8.17 Features and Fixes

Replace the conditional logic by proper dependency specification syntax. by @nielsn in #380

[soda-library] 1.8.16

^{29 January 2025}

1.8.16 Features and Fixes

Update soda-dbt to support dbt artifacts from dbt-core >=1.5, <2.0 by @adkinsty in #375
SAS-5475 - fix CVE on duckdb and python-mysql-connector by @dakue-soda in #377
Restore support for Python 3.8 by @nielsn in #378
REALLY fix support for Python 3.8 by @nielsn in #379

[soda-library] 1.8.14

^{28 January 2025}

1.8.14 Features and Fixes

Added fix for bug of NoneType by @teresama in #372
Run in managed distributed mode when scan type is managedAgentDistributed. by @jzalucki in #371
Failed rows: fix check value if samples disabled by @m1n0 in #373

[soda-library] 1.8.13

^{09 January 2025}

1.8.13 Features and Fixes

Revert of CLOUD-9009 by @jzalucki in #370

[soda-library] 1.8.12

^{08 January 2025}

1.8.12 Features and Fixes

Observability: Fix quoting in query parts by @m1n0 in #369
Logs: Add dataset/column info to Logs, support in obs by @m1n0 in #367

[soda-library] 1.8.10 - 1.8.11

^{07 January 2025}

1.8.10 - 11 Features and Fixes

Add support for numbers in identifier. by @jzalucki in #365
Move batched scan to scan execute method. by @jzalucki in #364
Move TIMEUNIT above IDENTIFIER_UNQUOTED. by @jzalucki in #368

[soda-core] 3.4.4

^{07 January 2025}

3.4.4 Features and fixes

Remove autoescape from jinja - allow use of special chars in variables. by @jzalucki in #2193
Add support for numbers in identifier. by @jzalucki in #2197

Refer to the Soda Core Release Notes for details.

[soda-library] 1.8.6 - 1.8.9

^{06 January 2025}

1.8.6 - 9 Features and Fixes

Profiling datetime fix after release by @teresama in #360
Removed time profiling. Added test for timestamp with timezone by @teresama in #356
Run partition detection algorithm during observability. by @jzalucki in #357
Chore: relax jinja autoescape by @m1n0 in #358
Feature: CLOUD 8690 implementation of date time profiling by @teresama in #336

[soda-agent] 1.1.37

^{12 December 2024}

1.1.37

Removes reported CVEs from the internally used dtzar/helm-kubectl image by replacing it for sodadata/soda-agent-utils.

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.36

^{10 December 2024}

1.1.36

Fixes a bug introduced in 1.1.35.

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.35

^{10 December 2024}

1.1.35

Releases Soda Library 1.8.9.

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.8.5

^{02 December 2024}

1.8.5 Features and Fixes

Pydantic: implicit v1 support for programmatic scans by @m1n0 in #352
Add include null to valid_count and invalid_count and percentage version. by @jzalucki in #351
Yaml: read and parse files thread-safe (#2188) by @m1n0 in #354

[soda-core] 3.4.3

^{02 December 2024}

3.4.3 Features and fixes

Add include null to valid_count and invalid_count and percentage version. by @jzalucki in #2186
Yaml: read and parse files thread-safe by @m1n0 in #2188

Refer to the Soda Core Release Notes for details.

[soda-core] 3.4.2

^{28 November 2024}

3.4.2 Features and fixes

Foreach: Resolve vars in queries by @m1n0 in #2183
Chore: Use jinja sandbox for templates by @m1n0 in #2185

Refer to the Soda Core Release Notes for details.

[soda-library] 1.8.4

^{27 November 2024}

1.8.4 Features and Fixes

Fix profiling. by @jzalucki in #350

[soda-library] 1.8.3

^{26 November 2024}

1.8.3 Features and Fixes

BROKEN RELEASE - DO NOT USE

Run all prerequisite jobs before performance testing. by @jzalucki in #341
Reference check: support identity by @m1n0 in #345
Fix doc link for soda-scientific package by @dirkgroenen in #347
First implementation of batch queue logger. by @jzalucki in #348

[soda-agent] 1.1.34

^{19 November 2024}

1.1.34

This release maps to Soda Library 1.8.2.

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.33

^{15 November 2024}

1.1.33

This release maps to Soda Library 1.8.2.

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.8.2

^{14 November 2024}

1.8.2 Features and Fixes

Send mininum profiling information for observability scan. by @jzalucki in #344
Chore: Use jinja sandbox for templates by @m1n0 in #343

[soda-library] 1.8.1

^{13 November 2024}

1.8.1 Features and Fixes

Pydantic: Fix v1 support by @m1n0 in #340
Foreach: Resolve vars in queries by @m1n0 in #342

[soda-library] 1.8.0

^{29 October 2024}

1.8.0 Features and Fixes

CLOUD-8739: orchestrate observability by @jzalucki in #339

[soda-agent] 1.1.32

^{29 October 2024}

1.1.32

This release maps to Soda Library 1.6.4.

CLOUD-8265: Add ability to refer to existing secret containing API Key by @dirkgroenen in #104
CLOUD-8558: Secure execution of Soda Agent with least-privilege in mind by @dirkgroenen in #105

Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.7.1

^{25 October 2024}

1.7.1 Features and Fixes

Anomaly Detection check: add alert_directionality
Comparison row count check: secondary datasource filter fix (#2165) by @m1n0 in #337
Reconciliation row: fix multiple checks with filters by @m1n0 in #338

[soda-core] 3.4.1

^{22 October 2024}

3.4.1 Features and fixes

Add documentation for MS fabric package install + config by @janet-can in #2180
Fix: Comparison row count check secondary datasource filter by @asantoz in #2165

Refer to the Soda Core Release Notes for details.

[soda-core] 3.4.0

^{21 October 2024}

3.4.0 Features and fixes

Add support for Azure SQL, Synapse, and Microsoft Fabric and extend support for SQL Server by @sdebruyn in #2160
Add page to docs folder for data contracts language reference by @janet-can in #2166

Refer to the Soda Core Release Notes for details.

[soda-library] 1.7.0

^{17 October 2024}

1.7.0 Features and Fixes

Support both pydantic v1 and v2 by @m1n0 in #328

[soda-library] 1.6.5

^{17 October 2024}

1.6.5 Features and Fixes

Feature: changed auto_exclude_anomaly under TrainingDatasetParameters by @teresama in #333
Chore: performance testing CI pipeline

[soda-cloud] Bulk-edit dataset responsibilities and attributes

^{17 October 2024}

Soda’s newest feature enables you to apply attributes, or assign users or user groups, and set their permissions, for more than one dataset at a time. To do so, access the Datasets page in the UI, select multiple datasets, then select Edit. Changes you make apply to all the selected datasets.

[soda-agent] 1.1.31

^{16 October 2024}

1.1.31

This release maps to Soda Library 1.6.4. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.30

^{09 October 2024}

1.1.30

This release maps to Soda Library 1.6.4. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.6.4

^{08 October 2024}

1.6.4 Features and Fixes

Chore: Changed pandas version to be compatible with python 3.8 by @teresama in #323
Add tracing to bunch of classes + allow 1.6.4.dev0 by @jzalucki in #325
Update obs extreme values test after pandas2 by @m1n0 in #329
Run perf nightly, add more dd tags, fix none hostname. by @jzalucki in #330
Feature: offer user control to automatically exclude classified anomalies from training auto_exclude_anomalies in anomaly checks by @teresama in #327
Fix obs test for py38 by @m1n0 in #331
[CLOUD-8480] Revert “Put back global 10k query limit temporarily (#324)” by @dirkgroenen in #332

[soda-library] 1.6.3

^{26 September 2024}

1.6.3 Features and Fixes

Put back global 10k query limit temporarily by @m1n0 in #324

[soda-agent] 1.1.29

^{26 September 2024}

1.1.29

This release maps to Soda Library 1.6.3. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-cloud] Introducing custom roles

^{26 September 2024}

To manage the actions of users that belong to a single organization, Soda Cloud uses roles, groups, and access permissions. These roles and groups and their associated permissions enforce limits on the abilities for users to access or make changes to resources, or to make additions and changes to organization settings and default access permissions.

New in Soda Cloud, you can create and edit new global or dataset roles to customize access to resources and functionalities in your organization.

See: Manage global roles, user groups, and settings See: Manage dataset roles

[soda-agent] 1.1.28

^{25 September 2024}

1.1.28

This release maps to Soda Library 1.6.2. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.6.2

^{24 September 2024}

1.6.2 Features and Fixes

Remove global hard limit on queries by @m1n0 in #321
Fix: exclude NaNs (as NULLS) from aggregate queries in databricks by @bastienboutonnet in #322

[soda-agent] 1.1.27

^{18 September 2024}

1.1.27

This release maps to Soda Library 1.6.1. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.6.1

^{17 September 2024}

1.6.1 Features and Fixes

Dataset level configuration for attributes and samples columns. by @jzalucki in #313
Do not collect samples if collecting of default samples were disabled in the cloud. by @jzalucki in #314
Use default cloud samples columns. by @jzalucki in #315
Fix: Handle cases where database returns NaN instead of NULL in aggs and frequent values queries by @bastienboutonnet in #316
Add support for collect failed rows table and checks level. by @jzalucki in #317
CLOUD-8251 - Fix Oracle in CI by @dakue-soda in #302
Add custom message to DefaultSampler depending on samples disabled reason. by @jzalucki in #319
Send soda library version during file upload, if fileId not present mark sample as not persisted with a message. by @jzalucki in #318

[soda-core] 3.3.15 - 22

^{12 September 2024}

3.3.15 - 3.3.22 Fixes

Removing the data source name lower case requirement by @tombaeyens in #2161
Fixing Spark session API by @tombaeyens in #2159
Fixing the lacking data source error message on contract build by @tombaeyens in #2158
Fixing test library dependencies for test table creation by @tombaeyens in #2157
Fixing atlan source contract yaml and lacking schema error message by @tombaeyens in #2153
Fixed the Atlan integration glue db-schema switch by @tombaeyens in #2152
Comparison check - fix other table filter by @jzalucki in #2149
Contracts7 : Fixing the integration correlation issue by @tombaeyens in #2148

Refer to the Soda Core Release Notes for details.

[soda-library] 1.6.0

^{04 September 2024}

1.6.0 Features and Fixes

Comparison check: Fix “other” table filter by @jzalucki in #304
Oracle: Bug fixes by @m1n0 in #260
Failed rows: Always expose failing sql for failed rows and user defined metric if failing query is available. by @jzalucki in #305
Failed rows: Add column property to failed rows and user defined metric checks. by @jzalucki in #307
Failed rows: Templatize rerouted sample message. by @jzalucki in #310
Observability: Catch issues when orchestrating profiling by @m1n0 in #311
Observability: Fall back to 1M if partition fails by @m1n0 in #312
Observability: Set duplicate percentage to None when zero rows in partition by @jzalucki in #301
Observability: Warn user when 24h partition is empty. by @jzalucki in #306

[soda-agent] 1.1.26

^{04 September 2024}

1.1.26

This release maps to Soda Library 1.6.0. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.25

^{19 August 2024}

1.1.25

This release maps to Soda Library 1.5.25. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.25

^{14 August 2024}

1.5.25 Fixes

Spark: Replicate implicit ‘include all’ in profiling by @m1n0 in #300 and #303
Freshness: Support variables in thresholds by @m1n0

[soda-core] 3.3.14

^{14 August 2024}

3.3.14 Fixes

Cross row count check should support custom identity. by @jzalucki in #2139
Handle SQL exception nicely for failed rows and user-defined check. by @jzalucki in #2140
Spark: Send discovery data despite errors. by @jzalucki in #2142
Spark: Failed rows should not be limited to max 100 total results. by @jzalucki in #2143
Freshness: Support variables in thresholds by @m1n0 in #2146
Spark: replicate implicit ‘include all’ in profiling consistently wit… by @m1n0 in #2147

Refer to the Soda Core Release Notes for details.

[soda-library] 1.5.24

^{13 August 2024}

1.5.24 Fixes

Adapt anomaly detector outcome messages for observability by @bastienboutonnet in #296
Always clean DB even on GH. by @jzalucki in #295
Group evolution: fix group changes not being detected by @m1n0 in #297
Feature: smaller min confidence interval ratio by @bastienboutonnet in #298
Execute observability checks outside of regular flow. by @jzalucki in #299

[soda-agent] 1.1.24

^{13 August 2024}

1.1.24

This release maps to Soda Library 1.5.24. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.23

^{02 August 2024}

1.5.23 Fixes

Attempt to always show freshness even if last 24 hours partition does not return data. by @jzalucki in #291
Observability: Always add partition column to profiling result by @m1n0 in #293

[soda-agent] 1.1.23

^{02 August 2024}

1.1.23

This release maps to Soda Library 1.5.23. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.22

^{01 August 2024}

1.5.22 Fixes

Observability: minimize metadata retrieval, do not push data into dis… by @m1n0 in #282
Handle SQL exception nicely for failed rows and user-defined check. by @jzalucki in #286
Spark: send discovery data despite errors. by @jzalucki in #290
Quote column names during observability partition detection. by @jzalucki in #288
Spark: failed rows should not be limited to max 100 total results. by @jzalucki in #292

[soda-agent] 1.1.22

^{01 August 2024}

1.1.22

This release maps to Soda Library 1.5.22. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.21

^{31 July 2024}

1.5.21 Fixes

Add nchar, nvarchar and binary to text types for profiling. by @jzalucki in #281
CLOUD 8061: alias table names in sql queries by @jzalucki in #280
Oracle data source properties prefix should be None instead of “None” when no service name is provided. by @jzalucki in #283
Sqlserver: use appropriate aggregate methods to build queries by @jzalucki in #284
Cross row count check should support custom identity. by @jzalucki in #285
Copyedit on frequency detection error message by @janet-can in #287
Chore: update auto-assignments by @milanaleksic in #289

[soda-core] 3.3.13

^{29 July 2024}

3.3.13 Fixes

Always reset logger when new scan instance is created. by @jzalucki in #2136

Refer to the Soda Core Release Notes for details.

[soda-cloud] Sync user groups with IdP

^{25 July 2024}

Soda Cloud introduces a new feature that enables Admins to regularly one-way sync the user groups you have defined in Okta or Azure Active Directory into Soda Cloud via SSO integration.

Doing so obviates the need to manually create user groups in Soda Cloud that you have already defined in your IdP, and enables your team to select an IdP-managed user groups when assigning ownership access permissions to a resource, in addition to any user groups you may have created manually in Soda Cloud.

Learn more

[soda-library] 1.5.20

^{24 July 2024}

1.5.20 Fixes

Fix: make sure labelling incorrect anomalies always returns something by @bastienboutonnet in #265

[soda-agent] 1.1.21

^{24 July 2024}

1.1.21

This release maps to Soda Library 1.5.19. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.19

^{23 July 2024}

1.5.19 Fixes

Always reset logger when new Scan instance is created. by @jzalucki in #277
Use SHOW TABLES and SHOW VIEWS instead of spark session catalog API. by @jzalucki in #278
Fix: apply cast to numerical for ms sqlserver by @bastienboutonnet in #279

[soda-library] 1.5.18

^{22 July 2024}

1.5.18 Fixes

Use spark session catalog to get all table names including temporary views. by @jzalucki in #276

[soda-core] 3.3.12

^{18 July 2024}

3.3.12 Fixes

Scan context: support list keys in getter by @m1n0 in #2135

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.1.20

^{18 July 2024}

1.1.20

This release maps to Soda Library 1.5.17. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.17

^{17 July 2024}

1.5.17 Fixes

Reconciliation: support custom source/target query with deepdiff strategy by @jzalucki in #269
Observability: apply 1M rows limit with time partition. by @jzalucki in #270
Snowflake: support custom hostname and port (#2109) by @m1n0 in #271
Add sslmode support to postgres and denodo (#2066) by @m1n0 in #273
Add Scan Context to read/write data from/to a scan (#2134) by @m1n0 in #272
Better user provided queries sanitize. (#2131) by @jzalucki in #275

[soda-core] 3.3.11

^{17 July 2024}

3.3.11 Features and fixes

Date formats: improve date format regex by @pholser in #2128
Duckdb: fix schema check for db in file by @m1n0 in #2130
Snowflake: support custom hostname and port @whummer in #2109
Improve user provided query sanitization. by @jzalucki in #2131
Scan Context: read/write data from/to a scan by @m1n0 in #2134
Add sslmode support to postgres and denodo by @m1n0 in #2066
Sqlserver: add support for custom parameters. by @jzalucki in #2132

Refer to the Soda Core Release Notes for details.

[soda-library] 1.5.16

^{16 July 2024}

1.5.16 Fixes

Observability: get all metric history by @m1n0 in #268

[soda-agent] 1.1.19

^{16 July 2024}

1.1.19

This release maps to Soda Library 1.5.14. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.15

^{15 July 2024}

1.5.15 Fixes

Fix: cast SUM query to NUMERIC in BQ by @bastienboutonnet in #266
Fix: output outlier holidays even when no country holiday by @bastienboutonnet in #267

[soda-cloud] User groups

^{08 July 2024}

Create user groups in Soda Cloud to manage role-based permissions (Admin, Manager, Editor, Viewer) to resources. Once created, assign role-based permission to access a dataset to user groups, or assign user groups as alert notification rules recipients, and more.

Refer to Create custom user groups for details.

[soda-core] 3.3.10

^{08 July 2024}

3.3.10 Features and fixes

Bugfix for the Atlan integration when using soda-core & contracts

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.1.17 & 1.1.18

^{03 July 2024}

1.1.17 - 18

These releases map to Soda Library 1.5.13 and 1.5.14, respectively. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.14

^{02 July 2024}

1.5.14 Fixes

Http Sampler: Do not invoke when no failed rows by @m1n0 in #264
Missing Count: Fix sample query by @m1n0 in #264
Between threshold: Fix error when using variables by @m1n0 in #264
Profiling: Fix discovery metadata bug by @m1n0 in #263

[soda-library] 1.5.13

^{28 June 2024}

1.5.13 Fixes

Databricks: run tests in CI by @m1n0 in #239
Fix CI by @m1n0 in #261
Set minimum version of the freshness detector to 0.0.7 by @bastienboutonnet in #262

[soda-core] 3.3.7 & 3.3.8 & 3.3.9

^{28 June 2024}

3.3.7 - 9 Features and fixes

Contracts4 by @tombaeyens in #2116
Duplicate check: Remove unused aggregated query by @m1n0 in #2118
Updated readme by @janet-can in #2104
Oracle: Fix formats, freshness, other minor fixes by @m1n0 in #2106
Sampler: Do not invoke http endpoint if no failed rows present by @jzalucki in #2115
Missing count: Fix sample query by @jzalucki in #2114
Between threshold: Fix error when using variables by @jzalucki in #2113

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.1.15 & 1.1.16

^{28 June 2024}

1.1.15 - 16

These releases map to Soda Library 1.5.13. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.12

^{27 June 2024}

1.5.12 Fixes

SAS-3334 For duckdb do not use database as filter at all. by @jzalucki in #259
Fix: explictly construct country holiday df and concat with outliers by @bastienboutonnet in #256

[soda-agent] 1.1.14

^{27 June 2024}

1.1.14

This release maps to Soda Library 1.5.12. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.11

^{24 June 2024}

1.5.11 Fixes and features

Fix: leave warning bounds and only make level be pass when warn by @bastienboutonnet in #257
Fix: handle overflowing timestamps for nanosecond precision overflow issues by @bastienboutonnet in #258

[soda-agent] 1.1.13

^{24 June 2024}

1.1.13

This release maps to Soda Library 1.5.11. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.10

^{21 June 2024}

1.5.10 Fixes and features

CLOUD-7426 Add scan_time to http payload. by @jzalucki in #255
Feature: exclude outliers from training via holiday interface by @bastienboutonnet in #254

[soda-agent] 1.1.12

^{21 June 2024}

1.1.12

This release maps to Soda Library 1.5.9. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.9

^{20 June 2024}

1.5.9 Fixes and features

Oracle: fix profiling/discovery queries by @m1n0 in #253

[soda-core] 3.3.6

^{20 June 2024}

3.3.6 Features and fixes

CLOUD-7708 - Add Snowflake CI account to pipeline for soda-core by @dakue-soda in #2088
CLOUD-7400 - Improve memory usage by @dirkgroenen in #2081
Duplicate check: fail gracefully in case of error in query by @m1n0 in #2093
Bump requests and tox/docker by @m1n0 in #2094
Duplicate check: support sample exclude columns fully by @m1n0 in #2096
Spark: profiling support more text types by @m1n0 in #2099
Spark: profiling support more numeric types by @m1n0 in #2100
Oracle: fix profiling/discovery queries, add numeric profiling by @m1n0 in #2101

Refer to the Soda Core Release Notes for details.

[soda-library] 1.5.7 & 1.5.8

^{18 June 2024}

Fixes and features

Observability: Fix Anomaly Detection check history retrieval by @m1n0 in #246
Spark: profiling support more text types (#2099) by @m1n0 in #250
Feature: use fail_only flag instead of warning 0 by @bastienboutonnet in #249
Oracle: fix queries, profiling and other by @m1n0 in #251
Duplicate check: support sample exclude columns fully by @m1n0 in #241
Spark: profiling support more numeric types by @m1n0 in #242
Fix: make gap removal wait for at least 5 days and implement simpler thresholds by @bastienboutonnet in #243
Profiling: support casting numericals to large data type by @m1n0 in #244
Cloud: better error handling and logging by @m1n0 in #245

[soda-agent] 1.1.10 & 1.1.11

^{18 June 2024}

These releases map to Soda Library 1.5.7 & 1.5.8, respectively. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.6

^{10 June 2024}

Fixes and features

Freshness in obs: add log msg when no data by @m1n0 in #238
Obs: get all metadata only when enabled, make tests more robust by @m1n0 in #240

[soda-agent] 1.1.9

^{10 June 2024}

This release maps to Soda Library 1.5.7. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.8

^{07 June 2024}

This release maps to Soda Library 1.5.5. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.1.7

^{06 June 2024}

This release maps to Soda Library 1.5.5. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.5

^{05 June 2024}

Fixes and features

SAS-3519 CLOUD-7769: Correctly map statuses from remote scans by @dirkgroenen in #232
Bump requests and tox/docker by @m1n0 in #236
Feature: make all non critical error messages be warnings in profiling by @bastienboutonnet in #235
Feature: observability anomalies are considered correctly classified unless negative feedback given by @bastienboutonnet in #234
Duplicate check: fail gracefully in case of error in query by @m1n0 in #237

[soda-cloud] Soda AI

^{30 May 2024}

Announcing Soda AI, a suite of generative AI (GenAI) features made generally available, and available for preview upon request.

Soda AI SQL Assistant and Soda AI Regex Assistant Powered by OpenAI’s GPT-3.5 & GPT-4, the generative SQL and regular expression assistants available in Soda Cloud’s no-code checks helps you write the queries and expressions you can add to validity, missing, SQL failed rows, and SQL metric checks. The Soda AI SQL or Regex Assistants are enabled for new Soda Cloud accounts by default. If you do not wish to use them, navigate to your avatar > Organization Settings, then click to remove the check from the box for Enable SQL and Regex Assistants Powered By Powered by OpenAI. Read more in Write SodaCL checks.

Ask AI Assistant Powered, in part, by kapa.ai, the Ask AI Assistant in Soda Cloud answers almost any question you have about Soda, from how to integrate with other tools, to writing SodaCL, to addressing errors, and beyond. Request preview access to begin experimenting with the power of GenAI-driven data quality testing.

[soda-library] 1.5.4

^{29 May 2024}

Fixes and features

CLOUD-7751 - fix nightly CI pipeline, use Snowflake CI account config… by @dakue-soda in #229
Feature: use partition row count (via aggregates) and use in duplicate percent by @bastienboutonnet in #230

[soda-agent] 1.1.5 & 1.1.6

^{29 May 2024}

These releases map to Soda Library 1.5.4. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.3

^{28 May 2024}

Fixes and features

CLOUD-7400: Improve memory usage for Queries by @dirkgroenen in #227
CLOUD-7702: Add Snowflake CI account to pipeline for soda-library by @dakue-soda in #223
CLOUD-7725: use newer thrift 0.20.0 by @milanaleksic in #228

[soda-agent] 1.1.4

^{27 May 2024}

This release maps to Soda Library 1.5.2. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.2

^{24 May 2024}

Fixes and features

Observability: handle one row scenario by @m1n0 in #224
User defined metric check: support failed rows query by @m1n0 in #226

[soda-core] 3.3.5

^{24 May 2024}

Features and fixes

Failed rows: fix warn/fail thresholds for fail condition by @m1n0 in #2084
Upgrade to latest version of ibm-db python client by @Antoninj in #2076
User defined metric check: support fail query by @m1n0 in #2089

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.1.3

^{24 May 2024}

This releases maps to Soda Library 1.5.2. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-cloud] Anomaly Dashboard

^{23 May 2024}

Introducing Soda’s Anomaly Dashboard for observability. Use the dashboard driven by machine learning to get automated insights into basic data quality metrics for your datasets. Learn more

Supported by Soda deployments that use a self-hosted or Soda-hosted Agent to connect to data sources.

[soda-library] 1.5.1

^{22 May 2024}

Fixes and features

Fix float comparisons in tests by @m1n0 in #222
Observability: hash metric identities by @m1n0 in #225

[soda-agent] 1.1.2

^{22 May 2024} This releases maps to Soda Library 1.5.1. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.5.0

^{20 May 2024}

Fixes and features

Observability beta (behind feature flag) @m1n0 in #198

[soda-library] 1.4.10

^{17 May 2024}

Fixes and features

Failed rows: fix warn/fail thresholds for fail condition (#2084) by @m1n0 in #221
upgrade sqlparse version inside soda base package by @Antoninj in #220

[soda-library] 1.4.9

^{14 May 2024}

Fixes and features

CLOUD-7400 Stream query data through memory, reducing memory footprint by @dirkgroenen in #210

[soda-agent] 1.1.1

^{14 May 2024}

This releases maps to Soda Library 1.4.8. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.4.8

^{07 May 2024}

Fixes and features

CLOUD-7362: add base exception to error log messages cloud payload by @Antoninj in #212
Fix automated monitoring, prevent duplicate queries by @m1n0 in #90
Denodo: fix connection timeout attribute (#2065) by @m1n0 in #215
DB2: Update db2_data_source.py (#2063) by @m1n0 in #216
Update autoflake precommit by @m1n0 in #214
SAS-3361: upgrade to latest version of ibm-db python client by @Antoninj in #213
Hive: support scheme by @m1n0 in #217
Bump dev requirements by @m1n0 in #218

[soda-core] 3.3.3 & 3.3.4

^{07 May 2024}

Features and fixes

Fix automated monitoring, prevent duplicate queries by @m1n0 in #2075
Hive support scheme by @m1n0 in #2077
Bump deps by @m1n0 in #2079
Update autoflake precommit by @m1n0 in #2070
Contracts v3 by @tombaeyens in #2067

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.1.0

^{25 April 2024}

Fixes

CLOUD-7385: Update probes for Spring Boot by @s4got10dev in #67

This releases maps to Soda Library 1.4.7. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-core] 3.3.2

^{24 April 2024}

Features and fixes

Rename argument in set_scan_results_file method by @ozgenbaris1 in #2047
Dremio: support disableCertificateVerification option by @m1n0 in #2049
Denodo: fix connection timeout attribute by @m1n0 in #2065
DB2: support security option by @4rahulae in #2063

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.0.8, 1.0.9, 1.0.10

^{22 April 2024}

These releases map to Soda Library 1.4.7. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.4.7

^{10 April 2024}

Fixes and features

Rename argument in set_scan_results_file method (#2047)
Dremio: support disableCertificateVerification option (#2049)

[soda-agent] 1.0.7

^{10 April 2024}

This release maps to Soda Library 1.4.7. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.0.6

^{09 April 2024}

This release maps to Soda Library 1.4.6. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.4.5 & 1.4.6

^{04 April 2024}

Fixes and features

SAS-3165 Only reset sampler when originally SodaCloudSampler by @dirkgroenen in #207
Feature: enable new anomaly detection algo in group by checks by @bastienboutonnet in #208

[soda-cloud] Soda AI Assistants

^{28 March 2024}

Powered by OpenAI’s GPT-3.5 & GPT-4, the generative SQL and regular expression assistants available in Soda Cloud’s no-code checks helps you write the queries and expressions you can add to validity, missing, SQL failed rows, and SQL metric checks.

Soda’s SQL or Regex assistants are enabled for new Soda Cloud accounts by default. If you do not wish to use them, navigate to your avatar > Organization Settings, then click to remove the check from the box for Enable SQL and Regex Assistants Powered By Powered by OpenAI.

[soda-agent] 1.0.4 & 1.0.5

^{24 March 2024}

These releases map to Soda Library 1.4.5 & 1.4.6, respectively. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.0.3

^{24 March 2024}

This release maps to Soda Library 1.4.4. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.4.4

^{23 March 2024}

Fixes and features

Failed rows: fix warn/fail thresholds by @m1n0 in #204
Bump opentelemetry to 1.22 by @m1n0 in #205

[soda-core] 3.3.1

^{23 March 2024}

Features and fixes

Feature: improved wording and tooltip formatting in simulator by @bastienboutonnet in #2038
Failed rows: fix warn/fail thresholds by @m1n0 in #2042
Bump opentelemetry to 1.22 by @m1n0 in #2043
Bump dev requirements by @m1n0 in #2045

Refer to the Soda Core Release Notes for details.

[soda-library] 1.4.3

^{20 March 2024}

Fixes and features

Add missing import for type annotations backwards compatibility by @Antoninj in #196
Refactor: Parse access_url from dbt config for new multicell org by @bastienboutonnet in #194
Timestamp conversion fixes by @Antoninj in #200
SAS-2966 Remove scan reference exception throw in local mode by @dirkgroenen in #199
Add test for checks level attributes by @m1n0 in #201
Fix: Attribute handler timezone test by @m1n0 in #202
Feature: Better legend wording and nicer tooltip formatting by @bastienboutonnet in #203

[soda-core] 3.3.0

^{15 March 2024}

Features

Contracts 2nd iteration @tombaeyens in #2006

Soda Core 3.3.0 supports the newest, experimental version of soda-contracts. The new version introduces changes that may not be compatible with the previous experimental version of soda-contracts. To continue using the first version of soda-contracts without any adjustments, upgrade to Soda Core 3.2.4 for the latest in bug fixes and updates.

Refer to the Soda Core Release Notes for details.

[soda-core] 3.2.4

^{15 March 2024}

Fixes and features

Fix: Support attributes on multiple checks by @milanaleksic in #2032
Use dbt’s new access_url pattern to access cloud API by @bastienboutonnet in #2035

Refer to the Soda Core Release Notes for details.

[soda-library] 1.4.2

^{05 March 2024}

Fixes

Dremio: fix token support (#2028) by @m1n0 in #195

[soda-core] 3.2.3

^{05 March 2024}

Fixes and features

Feature: implement daily and monthly seasonality to external regressor … by @baturayo in #2027
Dremio: Fix token support by @m1n0 in #2028

Refer to the Soda Core Release Notes for details.

[soda-agent] 1.0.2

^{05 March 2024}

This release maps to Soda Library 1.4.2. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 1.0.1

^{05 March 2024}

This release maps to Soda Library 1.4.1. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-agent] 0.9.2

^{05 March 2024}

This release maps to Soda Library 1.4.2. Access Soda documentation for instructions to upgrade a Soda Agent helm chart to use the latest version of Soda Library.

[soda-library] 1.4.1

^{01 March 2024}