Add GatewayFailure: the provider not answering is not a tool failing
Some checks failed
CI / test (push) Failing after 4m37s
CI / fixture (push) Failing after 8s

terminationFor sent every gateway error that was not Refused or Timeout
to ToolFailure, because the enum had nowhere else to put it. On
2026-09-22 that was 131 of 318 production runs, and not one of them was
a tool failing: 56 were retired model ids, 25 an exhausted Anthropic
balance, 45 Groq's free-tier rate limit -- the only one still happening.
An operator reading the termination column saw "a tool is broken" for
two weeks while the actual answer was "we are not paying for capacity".

GatewayFailure is the seventh termination. Rate limited, request
rejected, credential refused and unreachable land there; Refused and
Deadline keep their own reasons; a non-gateway error is still the tool
layer's. A delegation whose subagent died at the gateway now carries
that reason up to the parent instead of reading as a tool call that
failed.

Migration 000016 widens the CHECK that 000006 chose precisely so this
would be a migration rather than an ALTER TYPE. Its down folds any
GatewayFailure rows back to ToolFailure BEFORE narrowing the constraint,
which is the order that works; verified up, down and up again on a
scratch database. Existing rows are left as they are -- the trajectory
entries still carry the gateway.* code for anyone reclassifying history.

The surface wording is the one termination where "try again" is honest
advice, since the dominant cause clears within a minute.

Full suite run against a real database, including the tests that skip
without one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
This commit is contained in:
2026-09-22 12:48:09 +05:30
parent b765495eb7
commit 5166fde764
10 changed files with 108 additions and 34 deletions

View File

@@ -0,0 +1,10 @@
-- Reverting narrows the vocabulary, so rows that used the seventh value are
-- folded back into ToolFailure FIRST — the classification they would have had
-- before 000016 — or the narrower CHECK cannot be re-added at all. This loses
-- the distinction the up migration introduced, which is what reverting means.
UPDATE agent_runs SET termination = 'ToolFailure' WHERE termination = 'GatewayFailure';
ALTER TABLE agent_runs DROP CONSTRAINT agent_runs_termination_check;
ALTER TABLE agent_runs ADD CONSTRAINT agent_runs_termination_check CHECK (termination IN (
'Completed', 'BudgetExceeded', 'Deadline',
'ConfirmationPending', 'ToolFailure', 'Refused'
));

View File

@@ -0,0 +1,22 @@
-- A seventh termination reason: GatewayFailure.
--
-- 000006 chose a CHECK over an enum type so that "a new termination reason
-- should be a migration, but not one that requires ALTER TYPE". This is that
-- migration.
--
-- Until now the model provider failing — rate limited, request rejected, key
-- refused, unreachable — was recorded as ToolFailure, because the enum had no
-- other place for it. On 2026-09-22 that was 131 of 318 production runs, of
-- which not one was a tool failing. The distinction is the difference between
-- "what did we break" and "what are we not paying for", and the column that
-- every dashboard and eval groups by could not make it.
--
-- Existing rows are left as they are. Rewriting history from the trajectory
-- text would be guesswork against a message format that has changed twice;
-- the trajectory entries still carry the gateway.* error code for anyone who
-- needs to reclassify the past.
ALTER TABLE agent_runs DROP CONSTRAINT agent_runs_termination_check;
ALTER TABLE agent_runs ADD CONSTRAINT agent_runs_termination_check CHECK (termination IN (
'Completed', 'BudgetExceeded', 'Deadline',
'ConfirmationPending', 'ToolFailure', 'Refused', 'GatewayFailure'
));