Webhooks

Receive HTTP callbacks when events happen in your workspace.

Event types

• prompt.created — new prompt version created
• prompt.updated — prompt template changed
• execution.completed — prompt execution finished
• execution.failed — execution errored
• evaluation.completed — eval run finished
• evaluation.regression — eval score dropped by threshold

Payload example

POST /your-webhook-endpoint
Content-Type: application/json
X-Signature: sha256=<hmac>

{
  "event": "evaluation.completed",
  "data": {
    "evaluation_id": "eval_abc123",
    "prompt_name": "support/greeting",
    "score": 0.87,
    "examples_evaluated": 100,
    "regressions": 3
  },
  "timestamp": "2026-04-14T10:30:00Z"
}

Signature verification

Every webhook includes an X-Signature header. Verify with HMAC-SHA256 using your webhook secret:

import hmac, hashlib

def verify(secret, body, signature_header):
    expected = hmac.new(
        secret.encode(),
        body.encode(),
        hashlib.sha256,
    ).hexdigest()
    return hmac.compare_digest(f"sha256={expected}", signature_header)

Retries

Failed deliveries (non-2xx response or timeout) are retried with exponential backoff: 1m, 5m, 30m, 2h, 6h. After 5 failures, the webhook is disabled and you're notified.