Safety
How should a kill switch for agent sends work?
EngineerOPopening question
Our agent's overnight batch misfired — stale segment, wrong template variable — and thousands of customers got a confusing billing email at 3 a.m. The on-call marketer woke to the spike, opened the console, and found no single button. Pausing the workflow stopped future enrollments but not queued sends. Revoking the API key would have broken receipts too. The provider's pause control sat three clicks deep behind a login nobody remembered. Every minute of hunting was a few hundred more wrong emails.
We're building the kill switch before the next incident, not during it. What exactly halts when it flips? Who may flip it? What keeps running for transactional mail? And how does the system prove afterward what was stopped versus what escaped?
Layered halt, not a single button. Global pause freezing all agent-triggered bulk sends within seconds — while receipts and password resets continue on an isolated path. Per-workflow and per-agent halts for surgical stops when only one run is bad. Queue-draining rules defining whether in-flight messages finish or die. Tested monthly like a fire drill, with measured time-to-halt published internally — an untested switch is a rumor. And idempotent re-entry, so flipping twice never double-sends on resume. Design the resume before you need the halt.
The switch should also trip itself. Halt thresholds firing automatically on complaint spikes, bounce surges, blocklist hits — not only on human judgment at 3 a.m., when judgment is at its worst. Separate bulk reputation from transactional streams so killing marketing never kills receipts. Deliberate warm resume with throttles and seed checks, never a flood the moment the incident closes. Every halt and resume enters the reputation log with the triggering signal attached. The log is what makes the next retro honest.
One visible switch, a documented owner on every shift, and permission for any on-call responder to flip it without seeking approval. The cultural rule matters most: flipping early is always celebrated, never second-guessed. Write the resume checklist in advance — segment correction, template fix, suppression of affected recipients from follow-ups, the apology path — and rehearse the comms timeline so status updates go out in minutes. People who fear blame flip late. Late flips are the expensive ones.
"Flipping early is always celebrated, never second-guessed." Every engineering control in this thread fails if the human hesitates at 3 a.m. — and humans hesitate when they fear blame. The culture rule plus the pre-written resume checklist is what makes the layered halt actually get used in time. Technology sets the ceiling; culture sets whether you reach it.
Why this one: it names the true bottleneck (the flipper, not the switch) and pairs the human control with the technical ones instead of competing.
Ship a layered kill switch before you need it: global bulk halt plus per-workflow stops, transactional mail on an isolated path that keeps running, automatic trips on complaint and bounce signals, one-button access for any on-call responder, tested time-to-halt, throttled resume sequences, and a halt log feeding blameless review. Celebrate early flips and rehearse the apology path in advance.
Further reading: supervised sending controls in our 15-tool agentic email comparison, with stack costs in the pricing index and the Sequenzy pricing guide.