Add GatewayFailure: the provider not answering is not a tool failing
terminationFor sent every gateway error that was not Refused or Timeout to ToolFailure, because the enum had nowhere else to put it. On 2026-09-22 that was 131 of 318 production runs, and not one of them was a tool failing: 56 were retired model ids, 25 an exhausted Anthropic balance, 45 Groq's free-tier rate limit -- the only one still happening. An operator reading the termination column saw "a tool is broken" for two weeks while the actual answer was "we are not paying for capacity". GatewayFailure is the seventh termination. Rate limited, request rejected, credential refused and unreachable land there; Refused and Deadline keep their own reasons; a non-gateway error is still the tool layer's. A delegation whose subagent died at the gateway now carries that reason up to the parent instead of reading as a tool call that failed. Migration 000016 widens the CHECK that 000006 chose precisely so this would be a migration rather than an ALTER TYPE. Its down folds any GatewayFailure rows back to ToolFailure BEFORE narrowing the constraint, which is the order that works; verified up, down and up again on a scratch database. Existing rows are left as they are -- the trajectory entries still carry the gateway.* code for anyone reclassifying history. The surface wording is the one termination where "try again" is honest advice, since the dominant cause clears within a minute. Full suite run against a real database, including the tests that skip without one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
This commit is contained in:
10
migrations/000016_gateway_failure_termination.down.sql
Normal file
10
migrations/000016_gateway_failure_termination.down.sql
Normal file
@@ -0,0 +1,10 @@
|
||||
-- Reverting narrows the vocabulary, so rows that used the seventh value are
|
||||
-- folded back into ToolFailure FIRST — the classification they would have had
|
||||
-- before 000016 — or the narrower CHECK cannot be re-added at all. This loses
|
||||
-- the distinction the up migration introduced, which is what reverting means.
|
||||
UPDATE agent_runs SET termination = 'ToolFailure' WHERE termination = 'GatewayFailure';
|
||||
ALTER TABLE agent_runs DROP CONSTRAINT agent_runs_termination_check;
|
||||
ALTER TABLE agent_runs ADD CONSTRAINT agent_runs_termination_check CHECK (termination IN (
|
||||
'Completed', 'BudgetExceeded', 'Deadline',
|
||||
'ConfirmationPending', 'ToolFailure', 'Refused'
|
||||
));
|
||||
22
migrations/000016_gateway_failure_termination.up.sql
Normal file
22
migrations/000016_gateway_failure_termination.up.sql
Normal file
@@ -0,0 +1,22 @@
|
||||
-- A seventh termination reason: GatewayFailure.
|
||||
--
|
||||
-- 000006 chose a CHECK over an enum type so that "a new termination reason
|
||||
-- should be a migration, but not one that requires ALTER TYPE". This is that
|
||||
-- migration.
|
||||
--
|
||||
-- Until now the model provider failing — rate limited, request rejected, key
|
||||
-- refused, unreachable — was recorded as ToolFailure, because the enum had no
|
||||
-- other place for it. On 2026-09-22 that was 131 of 318 production runs, of
|
||||
-- which not one was a tool failing. The distinction is the difference between
|
||||
-- "what did we break" and "what are we not paying for", and the column that
|
||||
-- every dashboard and eval groups by could not make it.
|
||||
--
|
||||
-- Existing rows are left as they are. Rewriting history from the trajectory
|
||||
-- text would be guesswork against a message format that has changed twice;
|
||||
-- the trajectory entries still carry the gateway.* error code for anyone who
|
||||
-- needs to reclassify the past.
|
||||
ALTER TABLE agent_runs DROP CONSTRAINT agent_runs_termination_check;
|
||||
ALTER TABLE agent_runs ADD CONSTRAINT agent_runs_termination_check CHECK (termination IN (
|
||||
'Completed', 'BudgetExceeded', 'Deadline',
|
||||
'ConfirmationPending', 'ToolFailure', 'Refused', 'GatewayFailure'
|
||||
));
|
||||
Reference in New Issue
Block a user