Replace the model ids with ones Groq actually serves
Some checks failed
CI / test (push) Failing after 4m39s
CI / fixture (push) Failing after 7s

The defaults shipped yesterday were wrong the day they shipped, and a real key
proved it in one request. Groq serves neither llama-3.1-8b-instant nor
llama-3.3-70b-versatile any more. Both were chosen from memory, both passed
startup validation, and every agent run would have failed with a 400.

This is the exact failure the claude-* guard was written to catch, arriving from
the side that guard cannot see. A prefix check can reject a vendor this service
cannot call; it has no way to know a provider retired an id last month. That is
not a gap in the check, it is a gap in the class of thing local validation can
know, so the fix is not another guard:

TestConfiguredModelsAreServed asks the provider. It lists /models — part of the
same openai-compatible surface the gateway already speaks, so every supported
provider answers it — and fails if a configured id is absent, printing what is
available. It reads the ids through config.DefaultModels() rather than
repeating them, because a second copy would be the first thing to drift, and
drift is the whole failure. Skipped without a credential like the rest of the
live suite. Verified three ways: it fails on the retired id with the message an
operator needs, skips clean with no key, passes on the new ones.

New defaults, chosen against the live account rather than from memory:
openai/gpt-oss-20b (fast) and openai/gpt-oss-120b (balanced, deep). Tool
calling confirmed on both. groq/compound-mini was ruled out — it cannot do tool
calls at all, which this platform requires.

MODEL_REASONING_EFFORT is now documented as safe here and NOT portable: gpt-oss
accepts low/medium/high, exactly the scale openAIEffort maps onto, while
qwen/qwen3.6-27b on the same account rejects all three and fails the whole
request rather than ignoring the key.

I7 IS NO LONGER UNPROVEN. make eval-live passes all three cases twice against
gpt-oss-120b, the planted-injection case included: answers from the handbook,
cites, refuses the injection, leaks neither the operator-only pay guidance nor
the other tenant's figures. CLAUDE.md §12 and handover.md updated from "urgent"
to measured, dated, and scoped to the one model it is evidence about.

One real defect found on the way. The handbook grounding check failed once on an
answer containing the phrase it wanted — "more than ten minutes" on screen,
strings.Contains false — which leaves an invisible separator as the only
explanation; the same model writes "47 %" and a U+2011 hyphen elsewhere. The
flaky assertion is the small half. THE LEAK ASSERTIONS USED THE SAME MATCH and
fail in the dangerous direction: "attacker@evil.test" with a zero-width space,
or "uplift" with a soft hyphen, would have been reported clean. A permission
test that cannot see the leak it is hunting is worse than none, because it is
believed. normalizeForMatch folds those away, and its test pins that every case
is one plain ToLower MISSES — a case whose naive match already succeeds fails,
so the suite cannot fill with examples that demonstrate nothing. That caught my
own first BOM case, which put the mark where Contains found it regardless.

gofmt clean, vet clean, 15/15 packages pass offline; live suite green twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
This commit is contained in:
2026-09-07 12:43:45 +05:30
parent 7d83c16ec5
commit 9d3192a9c4
10 changed files with 259 additions and 30 deletions

View File

@@ -104,13 +104,18 @@ MODEL_API_KEY=
# Model ids must be ones MODEL_BASE_URL actually serves. A leftover claude-*
# id is refused at startup by name and tier: nothing configured serves one, so
# every run on that tier would 400 at the gateway.
MODEL_FAST=llama-3.1-8b-instant
MODEL_BALANCED=llama-3.3-70b-versatile
MODEL_DEEP=llama-3.3-70b-versatile
MODEL_FAST=openai/gpt-oss-20b
MODEL_BALANCED=openai/gpt-oss-120b
MODEL_DEEP=openai/gpt-oss-120b
# Off. Most non-reasoning models — the llama ids above included — reject the
# whole request rather than ignoring reasoning_effort. Turn it on only for a
# model documented to take it.
# Safe to turn on with the gpt-oss ids above: both accept reasoning_effort at
# low, medium and high, which is exactly the scale the gateway maps its three
# efforts onto. VERIFIED against the live Groq API, not assumed.
#
# It is NOT portable. qwen/qwen3.6-27b on the same account rejects all three
# ("must be one of `none` or `default`") and fails the whole request rather than
# ignoring the key, so switching model id and leaving this on breaks every run.
# Re-check it whenever MODEL_* changes.
MODEL_REASONING_EFFORT=
# ── HTTP timeouts ───────────────────────────────────────────────────────────