Appearance
PagerDuty webhooks
PagerDuty is Exhale’s primary paging integration. See Incoming integrations for the full source roadmap and Manual webhook for custom JSON ingest.
Exhale accepts PagerDuty v3 webhook payloads at:
POST https://exhale.kolstromsystems.com/webhooks/pagerdutyDesign partners may use the staging host instead: https://exhale-staging.kolstromsystems.com/webhooks/pagerduty.
Response: 202 Accepted with pulse_id, status, created, and message. On replay dedupe (Duplicate webhook event), pulse_id may be null when no pulse row was created for that delivery.
OpenAPI: POST /webhooks/pagerduty — security scheme PagerDutySignature.
Required fields
event.data.id (PagerDuty incident id) is required. Payloads without it receive 400 Bad Request (do not use event.id as external_id). The body must be a JSON object. In staging/production, event.id (delivery id used for replay dedupe) is also required.
| Field | Source |
|---|---|
external_id | event.data.id |
title | event.data.title or event.data.summary |
event_type | event.event_type |
The full JSON body is stored after scrubbing (secrets and injection-like strings redacted).
Customer setup
Manual (account id + shared webhook)
- Sign in as admin → Connections → Integrations → PagerDuty.
- Enter your PagerDuty account id and save.
- Copy the shared webhook URL into PagerDuty (Generic Webhooks v3).
- Optionally enable Post scrubbed triage summary to PagerDuty incident notes and store a PagerDuty REST API token.
- Send a test alert or receive a signed webhook to mark the integration verified.
OAuth (one-click)
- Sign in as admin → Connections → Integrations → PagerDuty (OAuth).
- Click Connect with PagerDuty and approve the Exhale OAuth app.
- Exhale creates the webhook subscription and enables incident notes. Use Disconnect OAuth to tear down.
Manual and OAuth share one PagerDuty integration row; disconnect OAuth before switching back to manual account mapping.
Tenancy
Each pulse gets tenant_id from PagerDuty account mapping — Exhale matches account metadata in the webhook (account_id, event.account_id, or nested event.account.id) to your registered account id.
In staging and production, unmapped accounts are rejected:
- 404 when
account_idis present but not registered - 400 when account metadata is missing
- 403 when the mapped tenant is disabled or the trial has expired
- 503 when mapping is ambiguous or the platform is in maintenance
Local development may fall back to a dev tenant when no mapping matches.
Signature verification
Production and staging require X-PagerDuty-Signature: HMAC-SHA256 of the raw request body, formatted as v1=<hex>. Multiple comma-separated signatures are supported.
Unsigned or invalid requests receive 401 Unauthorized.
bash
SECRET="your-signing-secret"
BODY='{"event":{"id":"evt_demo_001","event_type":"incident.triggered","data":{"id":"P12345","title":"API latency is above SLO"}}}'
SIG="v1=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')"
curl -i -X POST "https://exhale.kolstromsystems.com/webhooks/pagerduty" \
-H "Content-Type: application/json" \
-H "X-PagerDuty-Signature: $SIG" \
-d "$BODY"Dedupe and reliability
Replay dedupe: When Redis is available, Exhale claims the PagerDuty delivery id (event.id) before scrub/ingest. A duplicate within the configured TTL is acknowledged with 202 and message Duplicate webhook event without re-running ingest. Staging/production reject payloads missing event.id with 400. If Redis is down and fail-closed mode is enabled (staging/production default), deliveries may receive 503.
Incident upsert: Duplicate deliveries for the same PagerDuty incident id (event.data.id) update the existing pulse row and still return 202.
Body size: Bodies larger than the configured maximum (default 1 MiB) receive 413. Invalid Content-Length receives 400. Staging/production also reject a missing Content-Length with 400.
Rate limits: Per-IP limits apply when configured (429 with Retry-After and X-RateLimit-* headers). Staging/production require rate limiting to be enabled.
PagerDuty v3 signs the raw body only — there is no timestamp in the signature header. Rely on event.id dedupe for replay protection.
Other event types
Acknowledged and resolved events update pulse status without re-enqueueing triage. Include event.id in staging/production:
bash
SECRET="your-signing-secret"
BODY='{"event":{"id":"evt_demo_ack_001","event_type":"incident.acknowledged","data":{"id":"P12345","title":"API latency"}}}'
SIG="v1=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')"
curl -i -X POST "https://exhale.kolstromsystems.com/webhooks/pagerduty" \
-H "Content-Type: application/json" \
-H "X-PagerDuty-Signature: $SIG" \
-d "$BODY"event.event_type | Pulse effect |
|---|---|
incident.triggered (and similar trigger types) | Upsert pulse; enqueue triage when async triage is enabled |
incident.acknowledged | Status → acknowledged |
incident.resolved | Status → resolved |
Triage write-back (incident notes)
When enabled in Connections → Integrations and a PagerDuty REST API token is stored, Exhale posts a scrub-safe plain-text summary to the incident after triage completes (POST /incidents/{id}/notes on PagerDuty). Delivery is async via the notification queue.