Appearance
Prometheus Alertmanager
Prometheus Alertmanager webhook receivers POST alert groups to Exhale. Each tenant gets a unique HTTPS URL token. Exhale scrubs the payload, upserts a Pulse (
source=alertmanager), and runs triage when status is firing / alerting.
Connect in the product: Connections → Sources → Observability & alerts → Prometheus Alertmanager.
On the hosted app, use the URL Connections copies. It starts with https://exhaleoncall.com/webhooks/. The customer host does not publish /v1/webhooks/....
Endpoint
POST https://exhaleoncall.com/webhooks/alertmanagerHeader: X-Exhale-Webhook-Token: {token}
Or path form:
POST https://exhaleoncall.com/webhooks/alertmanager/{token}Unknown tokens return 404. Disabled or expired-trial tenants return 403.
Customer setup
- Sign in as admin → Connections → Sources → Observability & alerts → Prometheus Alertmanager.
- Click Connect Alertmanager and copy the webhook URL.
- In Alertmanager, add a receiver with
webhook_configs.urlpointing at theExhale URL (or the header-only route plus
X-Exhale-Webhook-Token). - Route the alerts you care about to that receiver.
- Send test alert creates a synthetic pulse and marks the integration configured.
See Prometheus's official guide: Configuration.
Rotate webhook token replaces the previous URL right away. Update Alertmanager after rotating.
Required payload fields
| Field | Required | Notes |
|---|---|---|
groupKey | Yes | Stable id for dedupe (source=alertmanager) |
status | No | firing / alerting → triage; resolved / ok → resolved |
Lifecycle mapping
Alertmanager status | Pulse status |
|---|---|
firing, alerting | Received → triage |
resolved, ok | Resolved |
Example
bash
curl -i -X POST "https://exhaleoncall.com/webhooks/alertmanager/${TOKEN}" \
-H "Content-Type: application/json" \
-d '{"version":"4","status":"firing","groupKey":"{}:{alertname=\"HighCPU\"}","title":"HighCPU"}'Auth, errors, and security
- Authenticate with header
X-Exhale-Webhook-Token(preferred) or the path/{token}form. Alertmanager has no vendor HMAC for these receivers. - Unknown tokens return 404. Disabled or expired-trial tenants return 403. Bodies over the size limit return 413. Missing
groupKeyreturns 400. - Successful ingest is 202. Redis delivery dedupe is per tenant (
groupKey+statuswithin the TTL). Exhale scrubs the body before persist. Rotate the webhook token in Connections → Sources if the URL leaks; the previous token stops working right away.