Skip to content

alert-triage

Alert triage — real, not ours, duplicate or resolved

Per new alert issue (a webhook, or an hourly sweep of untriaged ones): gathers live evidence, recent changes and likely duplicates, decides real / not ours / duplicate / resolved / noise with a cited cause, fix and owner, checks that on a calibrated classifier, and comments. Closing and relabelling wait for a person’s approval.

Source templates/alert-triage/workflow.yaml
Category debugging-ops
Trigger manual (wfx run) · schedule 23 * * * * · repository_dispatch alert_issue_opened
Steps 8
Human approval before act
Providers named claude-cli
Terminal window
wfx templates alert-triage
wfx new alert-triage --as my-alert-triage
wfx dryrun my-alert-triage

Triages alert issues: for a new one (repository_dispatch) or every untriaged one (hourly sweep), gathers read-only evidence — is the signal still there, what changed, which issues look the same — decides real / not ours / duplicate / resolved / noise with a cited cause, fix and owner, checks that verdict on a calibrated classifier and posts it as a comment. Closing or relabelling is only proposed; a person approves it.

  • env: ISSUE_REPO and ALERT_LABEL — the tracker and the label your watchers file under
  • gather step env: CHECKS_DIR / OBS_SSH / PROM_URL / LOKI_URL / LOKI_QUERY / DB_PSQL so ‘still happening’ can be re-checked — the same config as your prod-watch; copy its checks/ here
  • evidence.d/: add executables for your own evidence (runbooks, deploy history, a service’s health endpoint)
  • triage skills: add your product’s production-map skill (what is normal, which neighbours on shared servers are NOT yours)
  • the project env store needs TYPESAFE_API_KEY (or OPENROUTER_API_KEY) for the classifier
  • a sender for repository_dispatch: e.g. a GitHub Action on issues.labeled that POSTs {event_type: alert_issue_opened, client_payload: {issue: N}}
Input Type Default Required Meaning
issue integer 0
max_issues integer 5
dry_run boolean false
# Step Kind Skills Gates
1 gather
Pick issues and gather evidence
shell (run:) exitCode = 2 → fail
2 triage
Decide real / not ours / duplicate / resolved
agent
on claude-cli
prod-signal-triage
3 verdict
Check the triage (JEV)
shell + classifier (wfx judge) exitCode = 2 → fail
4 comment
Post the triage
shell (run:) exitCode = 1 → fail
5 pending
Anything to approve?
shell (run:)
6 unsure
Anything for a person?
shell (run:)
7 act
Close / relabel (approved)
shell (run:) human approval first
8 escalate
Ask a person about the unconfirmed ones
agent
on claude-cli
approval-desk
  • act waits for wfx approve <run-id> (or wfx reject <run-id> -m "why") before it runs.
  • escalate may call ask_human; answer with wfx answer <run-id> -m "...".