Link Search Menu Expand Document

Integrate webhooks with Soda Cloud

Configure a webhook in Soda Cloud to connect your account to a third-party service provider such as Jira, ServiceNow, PagerDuty, and more.

Use a webhook to:

  • send alert notifications for failed or warning check results to a third-party, such as ServiceNow
  • create and track data quality incidents with a third-party, such as Jira

webhook-example

Prerequisites
Configure a webhook
    Webhooks for Soda Cloud alert notifications
    Webhooks for Soda Cloud incident integrations
Example webhook with Jira for Soda Cloud incidents
Event payloads
Go further

Prerequisites

Configure a webhook

  1. Confirm that the third-party can provide an incoming webhook URL that meets the following technical specifications:
    • can return an HTTP status code between 200 and 400
    • can reply to a request within 10 seconds (otherwise the request from Soda Cloud times out)
    • provides an SSL-secured endpoint (https://) of TLS 1.2 or greater
  2. In your Soda Cloud account, navigate to your avatar > Organization Settings, then select the Integrations tab.
  3. Click the + at the upper right of the table of integrations to add a new integration.
  4. In the Add Integration dialog box, select Webhook then follow the guided steps to configure the integration. Reference the following tables for guidance on the values to input in the guided steps.
Field or Label Guidance
Name Provide a unique name for your webhook in Soda Cloud. Required
URL Input the incoming webhook URL or API endpoint provided by your service provider. See sections below for details. Required
alert notifications
incidents
HTTP Headers, Name For example, Authorization:
HTTP Headers, Value For example, bearer [token]
Enable to send notifications to this webhook when a check result triggers an alert. Check to allow users to select this webhook as a destination for alert notifications when check results warn or fail.
Use this webhook as the default notification channel for all check result alerts. Check to automatically configure check results alert notifications to this webhook by default.
Users can deselect the webhook as the notification destination in an individual check, but it is the prepopulated destination by default.
Enable to use this webhook to track and resolve incidents in Soda Cloud. Check to allow users to send incident information to a destination.
For example, a user creating a new incident can choose to use this webhook to create a new issue in Jira.


Webhooks for Soda Cloud alert notifications

You can use a webhook to enable Soda Cloud to send alert notifications to a third-party provider, such as OpsGenie, to notify your team of warn and fail check results. With such an integration, Soda Cloud enables users to select the webhook as the destination for an alert notification. See Edit checks in Soda Cloud.

Soda Cloud alert notifications make use of the following events:

Access a third-party service provider’s documentation for details on how to set up an incoming webhook or API call, and obtain a URL to input into the Soda webhook configuration in step 4, above. The following links may be helpful starting points.


Webhooks for Soda Cloud incident integrations

You can use a webhook to integrate with a third-party service provider, such as Jira, to track incidents. With such an integration, Soda Cloud displays an external link for the integration in the Incident Details. See image below for an example.

Soda Cloud incident integrations make use of the following events:

When Soda Cloud sends an incidentCreated event to a webhook endpoint, the third-party service provider can respond with a link message. In such a case, Soda Cloud adds the link to the incident. The following is an example of the response payload.

// > POST [webhook URL]
{
  "event": "incidentCreated",
  // ...
}
// < 200 OK
{
  "link": {
    "url": "https://sodadata.atlassian.net/browse/SODA-69",
    "text": "[SODA-69] Notification & Incident Webhook"
  }
}


For incident integrations with third-party service providers that do not provide a link message in the response, you can use a callback URL. In such a case, when Soda Cloud sends an incidentCreated event to the third-party, you can configure the third-party response to include an incidentLinkCallbackUrl property. See an example with Jira below.

Configure the third-party response to make a POST request to this callback URL, including the text and url in the body of the JSON payload. Soda Cloud adds the callback URL as an integration link in the incident details.

The following is an example of the response payload with a callback URL.

// > POST [webhook URL]
{
  "event": "incidentCreated",
  "incident": { ... },
  "incidentLinkCallbackUrl": "https://cloud.soda.io/integrations/webhook/8224bbc2-2c80-4c6d-a*****/incident-link/510fad8c-dc43-419a-a122-712a***/uLYosxWNwVGHSdR-_noJjlNAA--WyQwe1ygqGBg*****Q"
}
// < 200 OK
{ }
Followed by a POST request to incidentLinkCallbackUrl:
// > POST https://cloud.soda.io/integrations/webhook/8224bbc2-2c80-4c6d-a002-16***4e/incident-link/510fad8c-dc43-419a-a122-7***97/uLYosxWNwVGHSdR-_noJjlNAA--WyQwe1ygqGBg****IrQ
{
  "url": "https://sodadata.atlassian.net/browse/SODA-69",
  "text": "[SODA-69] Notification & Incident Webhook"
}


Example webhook with Jira for Soda Cloud incidents

In Jira, you can set up an Automation Rule that enables you to define what you want an incoming webhook to do, then provides you with a URL that you use in the URL field in the Soda Cloud integration setup.

This example offers guidance on how to set up a webhook to generate an external link to which Soda Cloud displays in the Incident Details. When you change the status of a Soda Cloud incident, the webhook also updates the status of the Jira issue that corresponds with the incident.

webhook-incident


In Jira, start by navigating to Project settings > Automation, then click Create rule and, for the type of New trigger, select Incoming webhook. Reference the Jira documentation for details on how to create an Incoming webhook.

The images below offer details for an example of an Automation Rule in Jira to set up an incoming webhook that processes incident events from Soda Cloud.

See full Jira Automation example


When configuring the Incoming webhook, note the value for the Webhook URL that Jira provides for the automation; input this URL into the Soda Cloud URL field for the webhook (step 4, above).

webhook-incoming

webhook-config


When configuring Send web request, note the Web request URL that Jira sends back to Soda Cloud for the incident. This example uses incidentLinkCallbackUrl to send a POST request back to Soda Cloud to display a link to the Jira issue in the Incident Details page.

webhook-send-request


When configuring a Branch rule to update the status of an existing Jira issue, note the value in the JQL field that identifies which issue to update when the incident status in Soda Cloud changes.

webhook-branch-rule


Event payloads

The following list of event payloads outlines the information that Soda Cloud sends when an action triggers a webhook.

validate

Soda Cloud sends this event payload to validate that the integration with the third-party service provider works. Soda Cloud sends this event during the guided workflow to set up an integration.

{
  "event": "validate",
  "sentAt": "2022-10-01T09:12:10.042323Z" 
}

checkEvaluation

Soda Cloud sends this event payload when it receives new check results. If the check is part of an agreement, the payload includes the agreement identifier.

{
  "event": "checkEvaluation",
  "checkResults": [
    {
      "id": "39d706c3-5a48-4f4b***",
      "sodaCloudUrl": "https://cloud.soda.io/checks/39d706c3-5a48-b",
      "definition": "checks for SODATEST_Customers_6f90f4ad:\ncount same as SODATEST_RAWCUSTOMERS_7275c02c in postgres2",
      "datasets": [
        {
          "id": "e8f1fe55-ae3c-44bd-",
          "sodaCloudUrl": "https://cloud.soda.io/datasets/e8f1fe55-ae3c",
          "name": "bnm_orders",
          "label": "bnm_orders",
          "tags": [],
          "owner": {
            "id": "31781df5-93cf-***",
            "email": "person@soda.io"
          },
          "datasource": {
            "id": "5a152025-26f6-",
            "name": "sodaspark",
            "label": "sodaspark"
          },
          "attributes": [
            {
              "id": "f0cd7b0f-4ac6-42a1-",
              "label": "Data Domain",
              "name": "data_domain",
              "value": "Product"
            },
            {
              "id": "32986775-3c7a-4a81-bfdb-5f9853746c39",
              "label": "Origin",
              "name": "origin",
              "value": "Pipeline"
            }
          ]
        }
      ],
      "column": "columnName",
      // pass, warn or fail
      "outcome": "pass",
      "dataTimestamp": "2022-01-04T09:49:48.060897Z",
      "diagnostics": {
        "value": 0.0,
      },
      // included when a check belongs to an agreement
      "agreement": {
        "id": "AGREEMENT-001-0000-0000-0",
        "sodaCloudUrl": "https://cloud.soda.io/agreements/AGREEMEN-T001-0000-0000-0",
        "label": "My new agreement pending",
        "approvalState": "pending",
        "evaluationResult": "warning"
      }
    }
  ]
}

incidentCreated

Soda Cloud sends this event payload when you create a new incident.

{
  "event": "incidentCreated",
  "incident": {
    "id": "e1f399a3-09ea-***",
    "sodaCloudUrl": "https://cloud.soda.io/incidents/e1f399a3-******-1992d2744ef6",
    "number": 196,
    "title": "Invalid customer ids",
    "description": "Invalid customer ids",
    "severity": "major",
    "status": "opened",
    "createdTimestamp": "2022-05-18T06:07:34Z",
    "lastUpdatedTimestamp": "2022-05-18T06:08:23Z",
    "resolutionNotes": "Stan is fixing the issue",
    "resolutionTimestamp": "2022-05-18T06:08:22.620196441Z",
    "links": [
      {
        "integrationType": "slack",
        "name": "soda-inc-196-2022-05-18-invalid-customer-ids",
        "url": "https://example.slack.com/channels/C03FU9GR7P7"
      }
    ],
    "lead": {
      "id": "31781df5-93cf-***",
      "email": "personn@soda.io"
    },
    "reporter": {
      "id": "31781df5-***",
      "email": "person@soda.io"
    },
    "checkResults": [
      // Contains the same payload as 
      // event checkEvaluation
    ]
  },
  "incidentLinkCallbackUrl": "https://cloud.soda.io/integrations/webhook/8224bbc2-******-16907465484e/incident-link/510fad8c-******-712a23f27197/uL******Kr6rvMcIrQ*"
}

incidentUpdated

Soda Cloud sends this event payload when an incident has been updated with, for example, a status change, when a new Lead has been assigned, or when check results have been added to the incident.

{
  "event": "incidentUpdated",
  "incident": {
    // Contains the same payload as 
    // event incidentCreated
  }
}

Go further

Was this documentation helpful?

What could we do to improve this page?


Last modified on 30-Sep-22