Notifications and Alerts

PACE can push notifications to Slack, Microsoft Teams, or email when key pipeline events occur. You configure channels (where to send) and alert rules (when to send).

Step 1 — Configure notification channels

Add a notifications: section to pace/pace.config.yaml:

notifications:
  slack:
    webhook_url: "${SLACK_WEBHOOK_URL}"
    channel: "#pace-alerts"          # optional override
  teams:
    webhook_url: "${TEAMS_WEBHOOK_URL}"
  email:
    smtp_host: "smtp.example.com"
    smtp_port: 587
    username: "${SMTP_USER}"
    password: "${SMTP_PASS}"
    from: "pace@example.com"
    to:
      - "team@example.com"
      - "oncall@example.com"

Store credentials in environment variables or CI secrets — never hardcode them in config.

Slack setup

1. Create an Incoming Webhook for your workspace.
2. Set SLACK_WEBHOOK_URL as a repository secret (GitHub) or environment variable.
3. The channel field is optional — it overrides the default channel set when you created the webhook.

Microsoft Teams setup

1. In Teams, go to the channel → Connectors → Incoming Webhook → Configure.
2. Copy the webhook URL and store it as TEAMS_WEBHOOK_URL.

Email setup

PACE sends email over SMTP with STARTTLS. Set smtp_port: 465 and use_ssl: true for SMTPS.

notifications:
  email:
    smtp_host: "smtp.gmail.com"
    smtp_port: 465
    use_ssl: true
    username: "${GMAIL_USER}"
    password: "${GMAIL_APP_PASSWORD}"
    from: "${GMAIL_USER}"
    to:
      - "team@example.com"

---

Step 2 — Configure alert rules

Add an alerts: section listing which events trigger which channels:

alerts:
  - event: hold_opened
    channels: [slack, email]
  - event: story_shipped
    channels: [slack]
  - event: cost_exceeded
    threshold_usd: 2.00
    channels: [slack]
  - event: pipeline_lock_timeout
    threshold_sec: 300
    channels: [slack]
  - event: update_available
    channels: [slack]

Event reference

Event	Fires when
hold_opened	GATE, SENTINEL, or CONDUIT issues a HOLD on any day
story_shipped	All agents issue SHIP and the day advances
cost_exceeded	FORGE story cost exceeds threshold_usd (USD)
pipeline_lock_timeout	A concurrent run cannot acquire the pipeline lock within threshold_sec seconds
update_available	A newer PACE version is found on the configured release channel

---

Notification message format

Each notification includes:

• Event type and timestamp
• Story title (for hold_opened and story_shipped)
• Hold reason (for hold_opened — taken from whichever agent issued the HOLD)
• Actual cost and threshold (for cost_exceeded)
• New version and changelog URL (for update_available)

Slack and Teams messages use card formatting. Email messages are plain text.

---

Testing notifications locally

export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/..."
python -c "
from pace.notifications import send_notification
send_notification('hold_opened', {
    'story': 'story-1',
    'reason': 'Test hold from local run'
})
"

---

Combining Slack and Jira

If you use platform.tracker: jira, escalation issues are opened in Jira automatically. You can still send Slack alerts for the same events — the two systems are independent.

platform:
  ci: github
  tracker: jira

notifications:
  slack:
    webhook_url: "${SLACK_WEBHOOK_URL}"

alerts:
  - event: hold_opened
    channels: [slack]