Krow/krow_backend
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove the Anthropic path; the gateway speaks one wire protocol
Some checks failed
CI / test (push) Failing after 4m38s
Details
CI / fixture (push) Failing after 7s
Details

Browse Source 
The platform now runs on Groq by default, through the OpenAI-compatible
chat-completions shape. That shape is not one vendor — Gemini, OpenRouter,
Together, vLLM and a local Ollama serve it too — so moving again stays
configuration rather than code.

Two things in the deleted file were not Anthropic's and would have gone
with it silently:

  withRetry / MaxAttempts / retryBackoff were defined in anthropic.go and
  CALLED BY openai.go. Deleting the file wholesale would have removed the
  retry policy of the provider that survived, and nothing in openai.go
  mentions it, so the loss would have been invisible until the next 429.
  The policy is a property of this platform's runs, not of a vendor's API;
  it now lives in retry.go where no provider can carry it off.

  StreamComplete had the same problem and moves to gateway.go, beside the
  Streamer interface whose comment already referenced it.

Three stale-configuration failures are now refused at startup instead of
being ignored. Each was verified firing through the real config.Load():

  MODEL_PROVIDER=anthropic — named separately from every other wrong value
  because it used to be correct. Ignoring it gives a stack that believes it
  is on Claude while every run goes to Groq and is billed there.

  ANTHROPIC_API_KEY set while MODEL_API_KEY is empty. Ignoring a key an
  operator did set is the worst version of this: they fail every run on a
  missing credential they are looking straight at.

  A leftover claude-* model id, naming the tier that carries it. This is
  the check the previous commit's error-detail work was diagnosing: such an
  id is accepted by this process, rejected by the provider, and 400s on
  EVERY run. "A model is wrong" does not say which of three lines to edit.

Defaults ship as a matched pair. defaultBaseURL and the three tier ids are
one decision, not four: an id is only meaningful against the service that
serves it, and a Groq id on an OpenAI base URL is the same failure from the
other side. The tiers also stop being one model — a tier whose cost does
not differ is a distinction that buys nothing.

Verified end to end against a stub of the wire, driving the real wiring
(config.Load in production mode, gateway.New, StreamComplete): streamed
deltas, tool-call decoding, the loopback credential exemption, and usage
totalling 150 rather than 190 — the cached-prefix subtraction still holds.

gofmt clean, go vet clean, 14/14 non-DB packages pass. httpserver still
needs a reachable database.

NOT verified: the I7 planted-injection eval. Removing this path removed the
only model whose refusal behaviour had been measured against it, so the new
default is unproven there until `make eval-live` runs with a real key. The
Groq model ids should also be confirmed against Groq's current lineup.
Flagged in CLAUDE.md §12 and docs/handover.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
This commit is contained in:
Suriyakumarvijayanayagam
2026-09-05 11:52:21 +05:30
parent cf99866e12
commit 34fa58a6b9
18 changed files with 391 additions and 765 deletions
Download Patch File Download Diff File Expand all files Collapse all files

115
go-api/internal/config/config.go
View File

@@ -19,9 +19,23 @@ import (
"time"
)
// defaultModel is what every reasoning tier routes to until a deployment says
// otherwise. Named once here so the three tiers cannot drift apart by accident.
const defaultModel = "claude-opus-5"
// The default model per tier, and the endpoint they are valid on.
//
// THESE THREE AND defaultBaseURL ARE ONE DECISION, not four. A model id is only
// meaningful against the service that serves it, so a Groq id with an OpenAI
// base URL is not a partial configuration — it is a broken one that starts
// cleanly and fails every run at request time. They changed together when the
// Anthropic path was removed and they have to keep changing together.
//
// Unlike the old single default, the tiers are no longer the same model: the
// point of a tier is that `fast` costs less than `deep`, and one id for all
// three made the distinction free and therefore meaningless.
const (
defaultBaseURL = "https://api.groq.com/openai/v1"
defaultFastModel = "llama-3.1-8b-instant"
defaultBalancedModel = "llama-3.3-70b-versatile"
defaultDeepModel = "llama-3.3-70b-versatile"
)
// Config is the whole of the Phase 1 configuration surface.
type Config struct {
@@ -36,8 +50,9 @@ type Config struct {
// KnowledgeConfig routes the retrieval layer's embedding provider.
//
// Anthropic does not serve embeddings, so the dense half of hybrid retrieval
// needs a separate credential. Voyage is the documented partner and the default.
// The chat provider does not serve embeddings, so the dense half of hybrid
// retrieval needs its own provider and credential — this is a separate choice
// from MODEL_*, and pointing one of them somewhere new does not move the other.
//
// An empty key is legitimate: this service boots and serves without one, and
// retrieval degrades to keyword-only rather than failing — reported on every
@@ -88,19 +103,20 @@ type KnowledgeConfig struct {
// first model call, as a structured gateway.not_configured a run can end with,
// not at startup as a refusal to boot.
type ModelConfig struct {
// Provider names the wire protocol: "anthropic" or "openai". Empty means
// anthropic, so a deployment that predates the second provider keeps
// working with the environment it already has.
// Provider names the wire protocol. "openai" is the only one, and empty
// means it; "anthropic" is refused at startup rather than ignored, because
// a deployment still carrying it has not been told the path was removed.
//
// "openai" is not only OpenAI. Groq, Gemini's compatibility endpoint,
// OpenRouter, Together, vLLM and a local Ollama all serve that same shape,
// and BaseURL is what chooses between them.
// and BaseURL is what chooses between them — which is why one wire protocol
// is not the same thing as one vendor.
Provider string
APIKey string
// BaseURL points the OpenAI-compatible provider at a specific service.
// Ignored by the anthropic provider, which has one endpoint.
// BaseURL points the provider at a specific service. Empty means the
// default in defaultBaseURL, which the default model ids belong to.
BaseURL string
Fast string
@@ -265,16 +281,17 @@ func Load() (*Config, error) {
},
Model: ModelConfig{
Provider: strings.ToLower(strings.TrimSpace(os.Getenv("MODEL_PROVIDER"))),
// MODEL_API_KEY first, then the Anthropic-specific name. Two
// spellings because the second provider is not Anthropic and
// ANTHROPIC_API_KEY=<a Groq key> would be a lie an operator has to
// keep re-reading; the fallback keeps every existing deployment
// working without an edit.
APIKey: firstSet("MODEL_API_KEY", "ANTHROPIC_API_KEY"),
BaseURL: strings.TrimSpace(os.Getenv("MODEL_BASE_URL")),
Fast: withDefault("MODEL_FAST", defaultModel),
Balanced: withDefault("MODEL_BALANCED", defaultModel),
Deep: withDefault("MODEL_DEEP", defaultModel),
// One spelling. ANTHROPIC_API_KEY used to be accepted as a
// fallback and is now deliberately NOT read: with the Anthropic
// path gone it would name a vendor this service cannot call, and
// silently authenticating to Groq with a variable called
// ANTHROPIC_API_KEY is the kind of lie an operator has to keep
// re-reading. A stale one is caught at startup, not ignored.
APIKey: strings.TrimSpace(os.Getenv("MODEL_API_KEY")),
BaseURL: withDefault("MODEL_BASE_URL", defaultBaseURL),
Fast: withDefault("MODEL_FAST", defaultFastModel),
Balanced: withDefault("MODEL_BALANCED", defaultBalancedModel),
Deep: withDefault("MODEL_DEEP", defaultDeepModel),
// 16k keeps a non-streaming response inside the SDK's HTTP
// timeout. The loop raises it and switches to streaming when it
// needs a long answer; this is the ceiling for a single
@@ -341,27 +358,59 @@ const DeepestAgentDeadline = 120 * time.Second
// or a production install with no credential that fails one run at a time
// instead of once at startup.
func (c *Config) validateModel() error {
// "anthropic" is named separately from every other wrong value because it
// is the one that used to be correct. A deployment still carrying it is not
// a typo, it is a stack that has not been told the path was removed — and
// the silent alternative is a service that believes it is on Claude while
// every run goes to Groq and is billed there.
switch c.Model.Provider {
case "", "anthropic", "openai":
case "", "openai":
case "anthropic":
return fmt.Errorf("MODEL_PROVIDER=anthropic is no longer supported: the Anthropic " +
"path was removed and this service speaks only the openai chat-completions " +
"shape. Unset MODEL_PROVIDER (or set it to openai) and point MODEL_BASE_URL " +
"at your provider")
default:
return fmt.Errorf("MODEL_PROVIDER must be anthropic or openai, got %q", c.Model.Provider)
return fmt.Errorf("MODEL_PROVIDER must be openai (or empty, which means openai), got %q", c.Model.Provider)
}
// A credential under the old name is refused rather than ignored. Ignoring
// it produces the worst version of this failure: a deployment that set a
// key, sees no error, and fails every run on a missing credential it is
// looking straight at.
if os.Getenv("ANTHROPIC_API_KEY") != "" && c.Model.APIKey == "" {
return fmt.Errorf("ANTHROPIC_API_KEY is set but is no longer read, and MODEL_API_KEY is " +
"empty: the Anthropic path was removed. Rename the variable to MODEL_API_KEY " +
"— and if that value is an Anthropic key, replace it, because nothing here can " +
"call Anthropic any more")
}
// A local model needs no credential, and demanding one would make the
// zero-cost development path impossible to configure. Everything else does:
// a production deployment without a key fails every run at the gateway,
// which is a misconfiguration wearing a runtime error's clothes.
if c.AppEnv == "production" && c.Model.APIKey == "" && !isLoopback(c.Model.BaseURL) {
return fmt.Errorf("MODEL_API_KEY (or ANTHROPIC_API_KEY) is required when APP_ENV=production; " +
return fmt.Errorf("MODEL_API_KEY is required when APP_ENV=production; " +
"without it every agent run fails at the model gateway")
}
// A base URL is only read by the OpenAI-compatible provider. Setting one
// while on anthropic is a deployment that believes it has switched
// providers and has not — it would keep calling Claude and keep being
// billed for it, with nothing in the logs to say so.
if c.Model.BaseURL != "" && c.Model.Provider != "openai" {
return fmt.Errorf("MODEL_BASE_URL only applies when MODEL_PROVIDER=openai; "+
"it is set to %q but the provider is %q, so the base URL would be ignored "+
"and every run would still go to Anthropic", c.Model.BaseURL, providerName(c.Model.Provider))
// A model id left over from the Anthropic path. THIS IS THE CHECK THAT
// REPLACED the old "base URL set against the wrong provider" one, and it
// guards the same failure from the other side.
//
// It is not hypothetical. A `claude-*` id sent to an OpenAI-compatible
// endpoint is accepted by this process, rejected by the provider, and
// surfaces as a 400 on EVERY run — which is exactly the incident that made
// the gateway start carrying upstream error text in the first place. One
// loud failure at startup is worth more than one per run.
for _, m := range []struct{ key, id string }{
{"MODEL_FAST", c.Model.Fast},
{"MODEL_BALANCED", c.Model.Balanced},
{"MODEL_DEEP", c.Model.Deep},
} {
if strings.HasPrefix(strings.ToLower(m.id), "claude") {
return fmt.Errorf("%s is %q, but the Anthropic path was removed: no configured "+
"provider serves a claude model, so every run on this tier would fail at "+
"the gateway. Set it to a model id your MODEL_BASE_URL (%s) serves",
m.key, m.id, c.Model.BaseURL)
}
}
if c.Model.BaseURL != "" {
u, err := url.Parse(c.Model.BaseURL)
@@ -582,7 +631,7 @@ func isLoopback(raw string) bool {
// rather than showing an empty string an operator then has to interpret.
func providerName(p string) string {
if p == "" {
return "anthropic (the default)"
return "openai (the default)"
}
return p
}

80
go-api/internal/config/model_test.go
View File

@@ -18,14 +18,20 @@ func TestValidateModelProvider(t *testing.T) {
wantErr bool
}{
{
"unset provider is anthropic, which is what every existing deployment has",
"unset provider is openai, which is now the only implementation",
modelCfg("development", ModelConfig{}), false,
},
{"anthropic named explicitly", modelCfg("development", ModelConfig{Provider: "anthropic"}), false},
{"openai", modelCfg("development", ModelConfig{Provider: "openai"}), false},
{"openai named explicitly", modelCfg("development", ModelConfig{Provider: "openai"}), false},
{
"anthropic is refused rather than ignored — it used to be correct",
modelCfg("development", ModelConfig{Provider: "anthropic"}), true,
},
{"a typo is caught once at startup, not once per run",
modelCfg("development", ModelConfig{Provider: "openal"}), true},
{"a provider that does not exist", modelCfg("development", ModelConfig{Provider: "groq"}), true},
{
"a vendor name is not a provider: groq is reached through openai + a base URL",
modelCfg("development", ModelConfig{Provider: "groq"}), true,
},
} {
t.Run(tc.name, func(t *testing.T) {
err := tc.cfg.validateModel()
@@ -36,28 +42,27 @@ func TestValidateModelProvider(t *testing.T) {
}
}
// THE EXPENSIVE MISTAKE.
// THE STALE CONFIGURATION.
//
// A deployment that sets MODEL_BASE_URL and forgets MODEL_PROVIDER believes it
// has moved off Claude. It has not: the anthropic path has one endpoint and
// ignores the field entirely, so every run keeps going to Anthropic and keeps
// being billed there, with nothing in the logs to say so. The whole point of
// this change is cost, and that is the one misconfiguration that silently
// defeats it.
func TestBaseURLWithoutOpenAIProviderIsRefused(t *testing.T) {
err := modelCfg("development", ModelConfig{
BaseURL: "https://api.groq.com/openai/v1",
}).validateModel()
// This replaced a test called TestBaseURLWithoutOpenAIProviderIsRefused, which
// guarded the mirror image of the same mistake: while both providers existed, a
// base URL without MODEL_PROVIDER=openai meant a deployment that believed it had
// left Claude and had not. That failure is now impossible — there is nowhere
// else for a run to go — and the surviving one points the other way: a
// deployment that still names Anthropic, and must be told rather than silently
// re-pointed at a provider it never chose.
func TestTheRemovedProviderIsRefusedLoudly(t *testing.T) {
err := modelCfg("development", ModelConfig{Provider: "anthropic"}).validateModel()
if err == nil {
t.Fatal("a base URL on the anthropic provider was accepted; every run would still go to Anthropic")
t.Fatal("MODEL_PROVIDER=anthropic was accepted; the stack would silently run on another vendor")
}
for _, want := range []string{"MODEL_BASE_URL", "MODEL_PROVIDER=openai", "Anthropic"} {
for _, want := range []string{"MODEL_PROVIDER=anthropic", "no longer supported", "MODEL_BASE_URL"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the message does not mention %q:\n %v", want, err)
}
}
// The same URL with the provider set is exactly the intended configuration.
// The intended configuration is exactly what the message tells them to set.
if err := modelCfg("development", ModelConfig{
Provider: "openai", BaseURL: "https://api.groq.com/openai/v1",
}).validateModel(); err != nil {
@@ -65,6 +70,45 @@ func TestBaseURLWithoutOpenAIProviderIsRefused(t *testing.T) {
}
}
// A model id that outlived its provider.
//
// The expensive shape of this is not a typo, it is an UNCHANGED .env: the tier
// ids were claude-* for the whole life of the Anthropic path, and nothing about
// switching providers forces them to be revisited. Left unchecked the process
// starts clean and every single run fails at the gateway with a 400 — which is
// the incident that made the gateway start carrying upstream error text at all.
func TestClaudeModelIdsAreRefused(t *testing.T) {
base := ModelConfig{Provider: "openai", BaseURL: "https://api.groq.com/openai/v1",
Fast: "llama-3.1-8b-instant", Balanced: "llama-3.3-70b-versatile", Deep: "llama-3.3-70b-versatile"}
for _, tier := range []string{"MODEL_FAST", "MODEL_BALANCED", "MODEL_DEEP"} {
t.Run(tier, func(t *testing.T) {
m := base
switch tier {
case "MODEL_FAST":
m.Fast = "claude-opus-5"
case "MODEL_BALANCED":
m.Balanced = "claude-opus-5"
case "MODEL_DEEP":
m.Deep = "claude-3-5-sonnet-latest"
}
err := modelCfg("development", m).validateModel()
if err == nil {
t.Fatalf("%s kept a claude id and was accepted; every run on that tier would 400", tier)
}
// Naming the tier is the whole value: "a model is wrong" does not
// tell an operator which of three lines to edit.
if !strings.Contains(err.Error(), tier) {
t.Errorf("the message does not name the tier %q:\n %v", tier, err)
}
})
}
if err := modelCfg("development", base).validateModel(); err != nil {
t.Fatalf("a fully-migrated configuration was refused: %v", err)
}
}
func TestBaseURLMustBeAURL(t *testing.T) {
for _, raw := range []string{"api.groq.com", "ftp://x.test", "not a url", "://broken"} {
err := modelCfg("development", ModelConfig{Provider: "openai", BaseURL: raw}).validateModel()