Ask a second provider when the first one is busy

A free tier's ceiling is tokens per MINUTE, and one run can exceed a whole
minute's worth by itself: a three-call run measured 12,123 against a ceiling of
8,000. withRetry already fires three times and all three are refused, because
1.6 seconds of backoff does not buy back a minute's budget. The run ends
GatewayFailure and somebody reads "the model did not answer".

Retrying harder cannot fix a ceiling. Asking somebody else can — the ceilings
are per provider, so a second key is a second budget. Groq, Cerebras, Gemini,
Mistral and OpenRouter all serve the same chat-completions shape, which is why
this is a list of Configs and not a second implementation.

Configured as MODEL_FALLBACK_<n>_BASE_URL / _API_KEY / _FAST / _BALANCED /
_DEEP, numbered because five fields times three providers packed into one
delimited string is a parser nobody can read under pressure. Empty is the
ordinary case and returns the primary unwrapped, so a single-provider
deployment carries no wrapper and behaves exactly as before.

Failover is NOT unconditional, and the two guards are the design:

  - Only a transient failure moves. Error.Retryable() already draws that line
    for retries and it is the same line here. A 401 is this deployment's own
    credential and a 400 is a malformed request; both fail identically at every
    vendor, so trying three turns one visible fault into three invisible ones.

  - Only an unpinned conversation moves. ToolCall.Extra carries provider
    metadata echoed back verbatim — Gemini 3's thought signature — and a vendor
    rejects a follow-up that drops its own. A conversation carrying any belongs
    to whoever started it, so failover is available on the first model call,
    which is where a rate limit usually lands anyway.

Streaming falls over only before the first fragment: once text is in the
reader's window, a second provider would continue that sentence in a different
voice.

What this does not do, since the gap is where the next bug lives: it does not
make a run cheaper, does not raise any one ceiling, and does not help when every
provider is exhausted at once. It turns one busy provider into a slower answer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

go-api/internal/gateway/routing.go

@@ -72,6 +72,10 @@ type Config struct {
 	Balanced Routing
 	Deep     Routing
 
+	// Fallbacks are further providers to try, in order, when this one cannot
+	// answer. See failover.go for when that is sound and when it is not.
+	Fallbacks []Config
+
 	// MaxOutputTokens applies when a request does not set its own.
 	MaxOutputTokens int64
 
@@ -104,7 +108,26 @@ type Config struct {
 // correctness matters more than cost, which is a judgement an operator makes
 // about a deployment, not one an agent author makes about a page.
 func FromConfig(c config.ModelConfig) Config {
+	var fallbacks []Config
+	for _, f := range c.Fallbacks {
+		// Model ids default to the primary's. Usually wrong for a different
+		// vendor and deliberately not silently corrected: an id the endpoint
+		// does not serve answers invalid_request, which is a visible fault an
+		// operator can fix, where a guessed substitution would be an invisible
+		// one nobody asked for.
+		if f.Fast == "" {
+			f.Fast = c.Fast
+		}
+		if f.Balanced == "" {
+			f.Balanced = c.Balanced
+		}
+		if f.Deep == "" {
+			f.Deep = c.Deep
+		}
+		fallbacks = append(fallbacks, FromConfig(f))
+	}
 	return Config{
+		Fallbacks:           fallbacks,
 		Provider:            c.Provider,
 		APIKey:              c.APIKey,
 		BaseURL:             c.BaseURL,
@@ -123,7 +146,15 @@ func FromConfig(c config.ModelConfig) Config {
 // `gateway.New(gateway.FromConfig(...))` and should not learn a concrete type:
 // the next provider is a change here and nowhere else.
 func New(cfg Config) Gateway {
-	return NewOpenAI(cfg)
+	primary := NewOpenAI(cfg)
+	if len(cfg.Fallbacks) == 0 {
+		return primary
+	}
+	rest := make([]Gateway, 0, len(cfg.Fallbacks))
+	for _, f := range cfg.Fallbacks {
+		rest = append(rest, NewOpenAI(f))
+	}
+	return NewFailover(primary, rest...)
 }
 
 // routingFor resolves a tier against a table.