Pin a failover conversation on any tool call, not just a signed one

ace4db8 refused to move a conversation whose ToolCall carried Extra — provider
metadata echoed back verbatim, Gemini 3's thought signature being the case it
was written for. That test is wrong, and in the exact direction that breaks
production.

Extra is populated by the provider that ISSUED the call. A conversation begun
on Groq carries none at all, so it read as movable; moving it hands Gemini an
assistant turn holding a function call with no thought signature, which is the
400 that took the cluster down on 2026-09-22. An absent field meant "came from
somewhere that does not sign", and it was read as "safe to move".

So the test is the tool call, not the metadata: any ToolCalls or ToolResults in
the conversation pin it to whoever has been answering. Failover stays available
on the first model call of a run, which is where a rate limit lands anyway.

Found while configuring Groq primary with Gemini as the fallback — the exact
pairing that triggers it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 13:24:16 +05:30
parent ace4db8db6
commit d242329e50
2 changed files with 49 additions and 12 deletions

View File

@@ -100,24 +100,32 @@ func (f *failover) Stream(ctx context.Context, req Request, onDelta func(string)
// deployment's own credential. Failing over on those turns one provider's
// configuration error into every provider's, and buries the fault.
//
// 2. THE CONVERSATION MUST NOT BE BOUND TO ITS PROVIDER. ToolCall.Extra
// carries provider metadata echoed back verbatim — Gemini 3's thought
// signature is the known case, and it REJECTS a follow-up that does not
// return it. That metadata is meaningless to a different provider and its
// absence is fatal to the one that issued it, so a conversation that
// already carries any is pinned to whoever produced it. In practice this
// means failover is available on the first model call of a run, which is
// where a rate limit usually lands anyway.
// 2. THE CONVERSATION MUST CARRY NO TOOL CALL AT ALL. Not merely "no
// provider metadata" — ANY tool call pins the conversation, and the
// difference is a bug this got wrong first time round.
//
// The reasoning that failed: ToolCall.Extra carries provider metadata
// echoed back verbatim (Gemini 3's thought signature), so it looked
// sufficient to refuse only when Extra was present. But Extra is populated
// by the provider that ISSUED the call. A conversation begun on Groq
// carries no Extra at all, so it looked movable — and moving it hands
// Gemini an assistant turn containing a function call with no thought
// signature, which is exactly the 400 that took production down on
// 2026-09-22. The absent field was read as "safe to move" when it meant
// "came from somewhere that does not sign".
//
// So the test is the tool call, not the metadata. A conversation that has
// called a tool belongs to whoever has been answering it. Failover is
// available on the first model call of a run, which is where a rate limit
// lands anyway, and nowhere else.
func canFailOver(req Request, err error) bool {
var gwErr *Error
if !errors.As(err, &gwErr) || !gwErr.Retryable() {
return false
}
for _, m := range req.Messages {
for _, tc := range m.ToolCalls {
if len(tc.Extra) > 0 {
return false
}
if len(m.ToolCalls) > 0 || len(m.ToolResults) > 0 {
return false
}
}
return true

View File

@@ -94,6 +94,35 @@ func TestFailoverWillNotMoveAConversationBoundToItsProvider(t *testing.T) {
}
}
func TestFailoverWillNotMoveAConversationThatHasCalledAToolAtAll(t *testing.T) {
// The case the first version got wrong. A conversation begun on Groq
// carries NO provider metadata, so a rule keyed on ToolCall.Extra read it
// as movable — and handing Gemini a function call it never signed is the
// 400 that took production down on 2026-09-22. Any tool call pins the
// conversation, signed or not.
var calls []string
f := NewFailover(
&scripted{name: "groq", err: gwErr(CodeRateLimited, 429), calls: &calls},
&scripted{name: "gemini", calls: &calls},
)
_, err := f.Complete(context.Background(), Request{
Messages: []Message{
{Role: RoleUser, Text: "how many open positions?"},
{Role: RoleAssistant, ToolCalls: []ToolCall{{
ID: "c1", Name: "open_positions", Input: json.RawMessage(`{}`),
// No Extra: Groq does not sign. That is the trap.
}}},
{Role: RoleUser, ToolResults: []ToolResult{{CallID: "c1", Content: `{"open":15}`}}},
},
})
if err == nil {
t.Fatal("want the rate limit raised, not a second provider's answer")
}
if len(calls) != 1 {
t.Errorf("called %v; an unsigned tool call still pins the conversation", calls)
}
}
func TestFailoverWithNoFallbacksIsTheProviderItself(t *testing.T) {
var calls []string
p := &scripted{name: "groq", calls: &calls}