Round-trip provider metadata on tool calls; Gemini requires it

Gemini 3 models attach a thought signature to every function call
and reject the follow-up -- 400, "Function call is missing a
thought_signature in functionCall parts" -- when the assistant
message echoing that call does not carry it back. The gateway
rebuilt the assistant turn from id, name and arguments alone, so
every tool-using run on Gemini died on its second model call, after
a first call that looked perfectly healthy. Found when production
was pointed at Gemini on 2026-09-22; rolled back to Groq within
minutes.

ToolCall gains an opaque Extra field: the raw JSON of the wire's
extra_content, captured on both the streaming and non-streaming
paths and emitted verbatim on the next request. The gateway does
not read it and must not -- the point of one wire shape is that a
vendor's private fields pass through untouched. Absent stays
absent; no provider receives a null it never sent.

Also: Gemini wraps its error body in a one-element array, which
the message parser read as "no detail". The trajectory therefore
said only "the model rejected the request" where the body named
the missing signature outright. Unwrapped now, so the next
provider quirk is legible in the trajectory instead of costing a
day of proxy captures.

Verified end to end with the real gateway against real Gemini: a
three-turn tool-calling run completed and the proxy confirmed the
signature on every echoed call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
This commit is contained in:
2026-09-22 13:15:39 +05:30
parent 5166fde764
commit 797ee5f2d2
4 changed files with 166 additions and 3 deletions

View File

@@ -339,3 +339,96 @@ func TestBaseURLDefaultsAndTrimsSlash(t *testing.T) {
t.Errorf("endpoint = %q, want the trailing slash collapsed", got)
}
}
// THE FAILURE THIS EXISTS FOR: a provider that attaches private metadata to a
// tool call and refuses the follow-up without it. Gemini 3 does exactly this
// ("Function call is missing a thought_signature"), and a gateway that rebuilt
// the assistant turn from id, name and arguments alone killed every tool-using
// run on its second model call — after a first call that looked healthy.
//
// The round trip is tested end to end: the provider's extra_content on the
// response must reappear, byte for byte, on the next request's echo of that
// call. The gateway must not care what is inside it.
func TestToolCallProviderMetadataIsRoundTripped(t *testing.T) {
const sig = `{"google":{"thought_signature":"El4KXAFpFH0T4CM3"}}`
gw, captured := serve(t, func(w http.ResponseWriter, _ *oaiRequest) {
_, _ = io.WriteString(w, `{
"choices":[{"message":{"role":"assistant","tool_calls":[
{"id":"call_x","type":"function",
"function":{"name":"open_positions","arguments":"{}"},
"extra_content":`+sig+`}]},
"finish_reason":"tool_calls"}],
"usage":{"prompt_tokens":10,"completion_tokens":5}
}`)
})
resp, err := gw.Complete(context.Background(), ask("how many open positions?"))
if err != nil {
t.Fatalf("Complete: %v", err)
}
if len(resp.ToolCalls) != 1 {
t.Fatalf("got %d tool calls, want 1", len(resp.ToolCalls))
}
if string(resp.ToolCalls[0].Extra) != sig {
t.Fatalf("Extra = %s, want the provider's extra_content verbatim", resp.ToolCalls[0].Extra)
}
// Second turn: the loop echoes the assistant's call and adds the result.
// This is the request Gemini rejects when the signature is missing.
_, err = gw.Complete(context.Background(), Request{Tier: TierBalanced, Messages: []Message{
{Role: RoleUser, Text: "how many open positions?"},
{Role: RoleAssistant, ToolCalls: resp.ToolCalls},
{Role: RoleUser, ToolResults: []ToolResult{{CallID: "call_x", Content: `{"count":14}`}}},
}})
if err != nil {
t.Fatalf("second Complete: %v", err)
}
var echoed *oaiToolCall
for i := range captured.Messages {
if len(captured.Messages[i].ToolCalls) > 0 {
echoed = &captured.Messages[i].ToolCalls[0]
}
}
if echoed == nil {
t.Fatalf("the second request did not echo the assistant's tool call: %+v", captured.Messages)
}
if string(echoed.ExtraContent) != sig {
t.Errorf("echoed extra_content = %s, want %s", echoed.ExtraContent, sig)
}
}
// A provider that sends no metadata must not receive an "extra_content": null
// it never asked for. Absent stays absent.
func TestToolCallWithoutProviderMetadataOmitsTheField(t *testing.T) {
msgs := []Message{
{Role: RoleAssistant, ToolCalls: []ToolCall{{ID: "call_1", Name: "open_positions", Input: json.RawMessage(`{}`)}}},
}
raw, err := json.Marshal(encodeOpenAIMessages("", msgs))
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(raw), "extra_content") {
t.Errorf("extra_content was emitted for a call that had none: %s", raw)
}
}
// Gemini wraps its error in a one-element array. The detail must survive,
// because a bare "the model rejected the request" is the difference between a
// one-line diagnosis and a day of one.
func TestProviderErrorDetailSurvivesArrayEnvelope(t *testing.T) {
cases := map[string]string{
`{"error":{"message":"bare object"}}`: "bare object",
`[{"error":{"message":"array wrapped"}}]`: "array wrapped",
` [ {"error":{"message":"padded"}} ] `: "padded",
`{"message":"top level"}`: "top level",
`[]`: "",
`not json`: "",
}
for body, want := range cases {
if got := openAIErrorMessage([]byte(body)); got != want {
t.Errorf("openAIErrorMessage(%s) = %q, want %q", body, got, want)
}
}
}