Alerting

Welcome to the alerting documentation, here you’ll learn how to set up effective alerts and notifications for you team.

Creating alerts

Click a project’s “Alerts” menu item to view the alerts defined, then click “Add alert” to get started.

To configure an alert you’ll need to give it a descriptive name, a query used to filter events, an operator and event count threshold. The following example configuration alerts when one or more events matches error and program = "api" within a 5 minute window.

After setting up the alerting query and threshold you’ll need to notify your team. Depending on the size of your organization you may want to notify teams separately, such as a frontend or backend team. The previous example notifies the operations team when any errors originate from the “api” program.

If you want to notify your entire development team you could simply alert on the query string error which will report anything with the level above warning.

Muting alerts

Clicking the mute button in the alert listing will prevent the alert from running until it is unmuted.

Creating notifications

The alert creation page allows you to create a notification, however, they are separate entities and are managed by clicking the “Notifications” menu item. Currently the following integrations are supported:

• Email
• PagerDuty
• Slack
• SMS
• Webhooks

Email

The email alert notification integration allows you notify team members via email addresses.

PagerDuty

The PagerDuty alert notification integration lets you create complex on-call schedules with various forms of alerting, including SMS, phone calls and others.

Slack

The Slack alert notification integration allows you to keep your team informed by receiving alert messages in your Slack channel(s) by providing a Slack Webhook URL.

SMS

The SMS notification integration allows you to receive SMS text messages on your mobile phone.

Webhooks

The webhook integration allows you to create custom alert integrations by providing a URL for alerts. The URL you provide will receive a POST request for each alert triggered.

The following is an example webhook request payload for a single error performing monitor check log event:

{
  "project": {
    "id": "web_production",
    "name": "Web Production"
  },
  "alert": {
    "id": "1eM71gypfa9NLh7fl4WZ1LNAYcJ",
    "project_id": "web_production",
    "name": "Errors",
    "description": "All production errors",
    "severity": "error",
    "query": "level > warning",
    "operator": ">=",
    "threshold": 1
  },
  "events": [
    {
      "id": "1eM7jxcd1ZuEIGp6NmlQ9PmYkiw",
      "level": "error",
      "message": "error performing monitor check",
      "fields": {
        "aws": {
          "log": {
            "group": "/aws/lambda/vitals_check_processor",
            "stream": "2020/07/07/[110]2ba51fea382243e2bef29b32b3a0d24c"
          },
          "region": "eu-west-1"
        },
        "check_id": 16604,
        "user_id": 92038,
        "error": "timeout exceeded",
        "function": "check_processor"
      },
      "timestamp": "2020-07-07T11:02:16.995Z"
    }
  ],
  "triggered": true
}