Link Search Menu Expand Document

Release notes for Soda Core

[soda-core] 3.0.1

29 June 2022

  • Re-introduce Spark for the Docker image by @jmarien in #1458
  • Build: require strict prophet v1.0.0 in scientific library by @bastienboutonnet in #1459
  • Comment for pinned prophet version by @m1n0 in #1460
  • Fix the e parameter by @jmarien in #1461

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0

28 June 2022

This is the general availability release for Soda Core with Soda CL.

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0rc3 Beta

27 June 2022

  • Doc: add comment about ordinal_position ordering by @bastienboutonnet in #1428
  • Refactor: use filesystem abstractions in distribution check by @baturayo in #1423
  • Fix: distribution check athena compatibility by @bastienboutonnet in #1429
  • Feat: profile and discover view tables by @baturayo in #1416
  • Code style section in contrib docs by @m1n0 in #1432
  • Unify data source api, remove redundant code. by @m1n0 in #1433
  • Fix: support athena in column profiling by @bastienboutonnet in #1430
  • Column profiling metadata fix by @tombaeyens in #1431
  • Feat: Support profile columns inclusion/exclusion behaviour for Spark by @baturayo in #1437
  • CORE-63 Added relative percentage change over time by @tombaeyens in #1435
  • Feat: Raise a MissingBinsAndWeights exception if soda scan runs without distribution_reference present by @tituskx in #1421
  • Flatten data source configuration schema by @m1n0 in #1441
  • Fix: Suppress prophet’s pandas: frame.append deprecation warning by @tituskx in #1440
  • Feat: send outcome reason to cloud for anomaly detection and schema checks by @baturayo in #1390
  • Add private key and other extra params to snowflake by @m1n0 in #1446
  • Feat: refer to DROs by name by @tituskx in #1422
  • Change: rename the update command to update-dro as it better describes what the command is used for by @tituskx in #1444
  • Feat/fix: ensure empty bins for integer columns are not created and fix bin width derivation by @baturayo in #1447
  • Do not quote table names in for-each block by @m1n0 in #1449
  • Feat: add env based option to run tests on views by @vijaykiran in #1442

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0rc2 Beta

22 June 2022

  • feat: add wasserstein distance and PSI methods to distribution checks by @tituskx in #1395
  • CORE-24 New freshness syntax by @tombaeyens in #1400
  • Verify that in spark-df arrays & structs don’t break anything by @tombaeyens in #1397
  • feat: add column exclusion to profile columns by @bastienboutonnet in #1396
  • feat: log no threshold error during parsing and provide more informative error during check summary by @tituskx in #1401
  • CORE-44 Fixed some extra timestamps to utc by @tombaeyens in #1405
  • [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1407
  • SODA-23 table dataset rename by @tombaeyens in #1404
  • feat: send distribution check results to cloud so that they can be plotted by @tituskx in #1402
  • Update README to include support for Amazon Athena by @stuart-robinson in #1409
  • refactor: Refactor scan.py to remove code duplicates by @baturayo in #1391
  • Update CONTRIBUTING to stipulate that users fork the repo. by @janet-can in #1413
  • Core 70 clean test schemas by @tombaeyens in #1415
  • fix: hotfix for historic measurements having none values by @baturayo in #1418
  • CORE-26 Fix change over time results value parsing by @vijaykiran in #1419
  • CORE-57 improved exception handling when creating data source by @tombaeyens in #1411
  • Another approach for the Docker image for Soda Core by @jmarien in #1398
  • Added 5 random chars to CI schema names by @tombaeyens in #1424
  • Fix drop table statement in test suite by @m1n0 in #1425
  • SODA-44 Added Z to timestamps in soda cloud json by @tombaeyens in #1408
  • Added docs on running tests by @tombaeyens in #1426
  • Fix schema check title by @vijaykiran in #1427
  • fix: more useful profiling warnings by @bastienboutonnet in #1420
  • CORE-37 Fixed schema type comparison for BigQuery by @tombaeyens in #1410

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0rc1 Beta

08 June 2022

  • 1175 spark by @m1n0 in #1382

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b19 Beta

02 June 2022

  • fix: handle %.% in profile columns properly and other bugs by @bastienboutonnet in #1377
  • Fix: cope with cloud disabled samples. by @m1n0 in #1393
  • BQ: regex switch to ‘r’ instead of backslash escaping by @m1n0 in #1394

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b18 Beta

01 June 2022

  • Scientific package tests on Athena. by @m1n0 in #1374
  • Update OT with scan/check counts by @vijaykiran in #1386
  • feat: add ability to send dataset samples to soda cloud (SODA-284) by @baturayo in #1372
  • fix: typo in data source package import by @bastienboutonnet in #1387
  • 627 Added default sampler returning a sample that is is not persistent by @tombaeyens in #1385
  • feat: cap distribution check to 1M rows by default by @tituskx in #1379
  • refactor: clean up logging for anomaly detection by @bastienboutonnet in #1389
  • fix: avoid parsing DRO name in distribution check until fully implemented by @bastienboutonnet in #1388
  • Downgrade markupsafe dependency by @m1n0 in #1392

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b17 Beta

26 May 2022

  • Pin versions in core by @vijaykiran in #1383

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b16 Beta

26 May 2022

  • Fixing suffix scanning of configuration and check files by @tombaeyens in #1365
  • Refactored to actual table and actual column names by @tombaeyens in #1370
  • Send Soda Cloud logs by @tombaeyens in #1380
  • Prevent upload when no sample rows are present by @tombaeyens in #1378
  • refactor: inform when columns are skipped in profiling via logs by @bastienboutonnet in #1375

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b15 Beta

23 May 2022

  • refactor: remove darts dependency by @bastienboutonnet in #1362
  • refactor: remove code duplication in sodacl_parser by @baturayo in #1361
  • SODA-248 fixed change over time checks by @tombaeyens in #1366
  • Athena support by @m1n0 in #1367
  • Added for each schema check by @tombaeyens in #1368

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b14 Beta

19 May 2022

  • Add defaultDataSource to cloud payload by @vijaykiran in #1359
  • fix: provide docker image with soda-scientific packaged by @bastienboutonnet in #1355
  • Fixing data source validity error message by @tombaeyens in #1357
  • Added cython to the setup.py file by @tituskx in #1360
  • #1353 SODA-494 Fixing recursive loading of files by @tombaeyens in #1356

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b13 Beta

18 May 2022

  • 1237 samples2 by @tombaeyens in #1340
  • Getting disable samples from cloud config by @tombaeyens in #1348
  • Test anomaly detection for numeric metrics by @baturayo in #1349
  • Updated contributing, fixed logs and added hint in comment how to add… by @tombaeyens in #1350
  • Fix: Automated monitoring revert issues SODA-489 by @baturayo in #1351
  • Soda 159 - Test anomaly detection for nested metrics by @baturayo in #1354

Refer to the Soda Core Release Notes for details.


[soda-core] 3.0.0b12 Beta

16 May 2022

  • Fix date eu/us formats. by @m1n0 in #1334
  • 1237 samples by @tombaeyens in #1328
  • feat: column profiling by @bastienboutonnet in #1322
  • Switch to latest prophet by @vijaykiran in #1335
  • throw log error and return empty string if histogram assumption broken by @bastienboutonnet in #1337
  • Freshness send microseconds to cloud. by @m1n0 in #1338
  • Cloud: timestamps use seconds resolution by @m1n0 in #1339
  • Deleted docs folder by @tombaeyens in #1343
  • feat: implement automated monitoring executor/runner by @baturayo in #1323
  • feat: add table discovery by @bastienboutonnet in #1341
  • fix(profiling): allow null results in text column aggregates by @bastienboutonnet in #1344
  • Fix: update check identity in case of automated monitoring by @baturayo in #1346

Refer to the Soda Core Release Notes for details.


[soda-core] 0.0.1 Beta

22 March 2022

This release marks the launch, or first beta release, of Soda Core and Soda Checks Language.

Reference the Soda Core OSS and SodaCL documentation for information on how to use the new CLI tool and domain-specific language for reliability.


Last modified on 01-Jul-22