Skip to content

Splunk On-Call ​

Splunk On-Call (formerly VictorOps) Outgoing Webhooks POST incident lifecycle events to Exhale. Each tenant gets a unique HTTPS URL token. Exhale scrubs the payload, upserts a Pulse (source=splunk_oncall), and runs triage on triggered events.

This is Splunk On-Call paging ingest (creates Pulses). For Splunk observability context at triage time, see Splunk context.

Optional triage summaries post back as Splunk On-Call incident notes via the Public API (X-VO-Api-Id / X-VO-Api-Key).

See Sources for how Splunk On-Call fits with the other incoming webhooks.

On the hosted app, use the URL Connections copies. It starts with https://exhaleoncall.com/webhooks/. The customer host does not publish /v1/webhooks/....


Endpoint ​

After Connect in Connections → Sources → On-call & paging → Splunk On-Call:

Preferred (keeps tokens out of proxy access logs):

POST https://exhaleoncall.com/webhooks/splunk-oncall

Send header X-Exhale-Webhook-Token: {token}.

Path form (also supported):

POST https://exhaleoncall.com/webhooks/splunk-oncall/{token}
  • token — opaque URL segment; this token authenticates the delivery.
  • Response: 202 Accepted with pulse_id, status, created, and message.

Unknown tokens return 404. Disabled or expired-trial tenants return 403.


Customer setup ​

  1. Sign in as admin → Connections → Sources → Splunk On-Call.
  2. Click Connect Splunk On-Call and copy the webhook URL.
  3. In Splunk On-Call, create Outgoing Webhooks for incident triggered / acknowledged / resolved (or Any-Incident with an event field).
  4. Paste the Exhale URL, method POST, content type application/json, and the payload template below.
  5. Use Send test alert to mark the integration configured (this tests Exhale ingest, not that Splunk On-Call is sending yet).

See Splunk's official guide: Custom outbound webhooks in Splunk On-Call.

json
{
  "event": "triggered",
  "incident_number": "${{STATE.INCIDENT_NUMBER}}",
  "entity_id": "${{ALERT.entity_id}}",
  "entity_display_name": "${{ALERT.entity_display_name}}",
  "state_message": "${{ALERT.state_message}}",
  "alert_url": "${{ALERT.alert_url}}",
  "monitoring_tool": "${{ALERT.monitoring_tool}}",
  "message_type": "${{ALERT.message_type}}"
}

incident_number is required. Lifecycle mapping: triggered → receive + triage; acknowledged → acknowledged; resolved → resolved.

Triage notes ​

Under Configure, store API ID and API Key from Splunk On-Call Integrations → API, and enable note write-back. Exhale posts a scrubbed triage summary to POST /api-public/v1/incidents/{incidentNumber}/notes.

See Splunk's official guide: Splunk On-Call API.


Operator guide ​

See Errors & authentication for structured errors, 413 size limits, Redis delivery dedupe, and token rotation. Splunk On-Call deliveries authenticate with the Exhale URL token (header or path).

Exhale by Kolstrom Systems LLC