Make the shipped example envs ones that can actually start

The krow-2 deploy failed on the ANTHROPIC_API_KEY guard, which is the guard
doing its job. Checking what an operator hits *after* fixing it turned up two
older faults in the files they are told to copy — both predating the Groq
switch, both fatal at boot.

HTTP_WRITE_TIMEOUT shipped as 30s in .env.example, .env.docker.example and the
compose default, while validateWriteTimeout refuses anything at or under the
deep tier's 2m deadline. `cp .env.docker.example .env && docker compose up`
could not start. Now 180s. krow-2 never saw this because someone had already
overridden it in that environment.

.env.docker.example carried no model block at all, so a production stack built
from it is refused for a missing MODEL_API_KEY. Added, with the Groq defaults
and the reasoning-effort note (most non-reasoning models reject the request
rather than ignoring the key).

Neither was subtle. Both survived because the examples were prose to every test
in this package: the validator and the file documenting it had no mechanical
connection, so tightening one silently invalidated the other. That connection
is now TestShippedExampleEnvActuallyBoots, which parses each example and runs
Load() on it under the APP_ENV the file itself declares — production for the
docker one, development for the root one, each internally consistent. Verified
by mutation: reverting the timeout, removing the key line, and restoring a
claude-* id each fail it with the message an operator would see.

Go does not treat these files as test inputs, so an example-only edit can be
served a stale pass from the test cache. Noted in the test; use -count=1.

Also documented the upgrade path in handover.md, including the one thing
startup validation cannot catch: renaming ANTHROPIC_API_KEY to MODEL_API_KEY
without replacing the value boots fine and 401s on every run.

gofmt clean, go vet clean, 15/15 packages pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g

.env.example

@@ -16,7 +16,12 @@ LOG_LEVEL=info               # debug | info | warn | error
 HTTP_HOST=127.0.0.1
 HTTP_PORT=8080
 HTTP_READ_TIMEOUT=15s
-HTTP_WRITE_TIMEOUT=30s
+# 180s, not 30s. internal/config REFUSES TO START when this is below the deep
+# tier's 2m agent deadline: the server would abort the response mid-run and the
+# caller would see 502 from the proxy in front, a gateway error for something no
+# gateway did. 30s shipped here for a long time and was the cause of exactly
+# that incident. Anything at or under 2m0s is a container that will not boot.
+HTTP_WRITE_TIMEOUT=180s
 HTTP_IDLE_TIMEOUT=60s
 HTTP_SHUTDOWN_TIMEOUT=10s
 # Browser origins allowed to call this API cross-origin, comma-separated.

docs/handover.md

@@ -196,6 +196,31 @@ otherwise produce a service that boots cleanly and fails every agent run.
 
 The default with nothing set is Groq.
 
+### Upgrading a deployment that ran Claude
+
+A running stack does not migrate itself, and the first thing it does after this
+change is refuse to start:
+
+```
+ERROR fatal error="ANTHROPIC_API_KEY is set but is no longer read, and
+MODEL_API_KEY is empty: the Anthropic path was removed..."
+```
+
+That is the guard working. Two edits to the deployment's env fix it:
+
+1. `MODEL_API_KEY=<a Groq key>`
+2. Delete `ANTHROPIC_API_KEY` from the environment entirely.
+
+**Renaming the variable without replacing the value is the trap.** An
+`sk-ant-...` under the name `MODEL_API_KEY` passes every startup check — the
+process cannot tell one opaque string from another — and then fails every run
+with `the model credentials were refused` and Groq's own text. Startup
+validation catches the *shape* of a stale configuration, never a wrong secret.
+
+`ANTHROPIC_API_KEY` is still passed through in `docker-compose.yml` on purpose:
+a host that kept exporting it gets the loud failure above instead of a
+container that boots with no credential and fails one run at a time.
+
 ```bash
 # Groq (the default — base URL and ids below are what you get unset)
 MODEL_BASE_URL=https://api.groq.com/openai/v1

go-api/internal/config/example_env_test.go

@@ -0,0 +1,132 @@
+package config
+
+import (
+	"bufio"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// TestShippedExampleEnvActuallyBoots loads each example env exactly as an
+// operator would and asserts the result passes validation.
+//
+// THIS TEST EXISTS BECAUSE BOTH EXAMPLES SHIPPED A CONFIGURATION THAT COULD NOT
+// START. HTTP_WRITE_TIMEOUT was 30s in files an operator is told to copy, while
+// validateWriteTimeout refuses anything at or under the deep tier's 2m
+// deadline — so `cp .env.docker.example .env && docker compose up` failed at
+// boot. Separately, .env.docker.example carried no model block at all, which in
+// production is a second refusal for a missing MODEL_API_KEY.
+//
+// Neither was a subtle bug. Both survived because the examples were prose to
+// every test in this package: the validator and the file documenting it had no
+// mechanical connection, so tightening one silently invalidated the other.
+// That connection is this test.
+//
+// CAVEAT: `go test` does not treat these files as inputs, so a run that changes
+// ONLY an example env can be served a stale pass from the test cache. Verify
+// example edits with `-count=1`. `make test` and CI run from a clean cache and
+// are not affected.
+func TestShippedExampleEnvActuallyBoots(t *testing.T) {
+	for _, tc := range []struct {
+		path string
+		// Values an operator must supply, standing in for the placeholders the
+		// file ships. Only credentials and hostnames belong here — anything
+		// else would be this test papering over a broken example.
+		operatorSupplies map[string]string
+	}{
+		{
+			path:             filepath.Join("..", "..", "..", "infrastructure", ".env.docker.example"),
+			operatorSupplies: map[string]string{"MODEL_API_KEY": "gsk-operator-supplied"},
+		},
+		{
+			path:             filepath.Join("..", "..", "..", ".env.example"),
+			operatorSupplies: map[string]string{"MODEL_API_KEY": "gsk-operator-supplied"},
+		},
+	} {
+		t.Run(filepath.Base(tc.path), func(t *testing.T) {
+			env, err := parseDotenv(tc.path)
+			if err != nil {
+				t.Fatalf("reading %s: %v", tc.path, err)
+			}
+			for k, v := range tc.operatorSupplies {
+				env[k] = v
+			}
+			// Each file is validated under the APP_ENV IT DECLARES, not under
+			// one this test imposes. The two examples describe different
+			// deployments and each is internally consistent: .env.docker.example
+			// is production with sslmode=require, .env.example is development
+			// with sslmode=disable. Forcing production onto the development file
+			// fails it on a setting that is correct for what it is.
+			if env["APP_ENV"] == "" {
+				t.Fatalf("%s declares no APP_ENV; every example must say what it is", tc.path)
+			}
+
+			os.Clearenv()
+			for k, v := range env {
+				t.Setenv(k, v)
+			}
+
+			cfg, err := Load()
+			if err != nil {
+				t.Fatalf("%s cannot start: %v\n\n"+
+					"An operator copying this file gets this error, not a running service. "+
+					"Fix the example, not this test.", tc.path, err)
+			}
+			// Load() succeeding is the assertion. These guard the two specific
+			// regressions above, so a future edit that reintroduces either one
+			// fails by name rather than as a generic validation error.
+			if cfg.HTTP.WriteTimeout <= DeepestAgentDeadline {
+				t.Errorf("HTTP_WRITE_TIMEOUT is %s, which does not exceed the deep tier's %s deadline",
+					cfg.HTTP.WriteTimeout, DeepestAgentDeadline)
+			}
+			for _, m := range []struct{ key, id string }{
+				{"MODEL_FAST", cfg.Model.Fast},
+				{"MODEL_BALANCED", cfg.Model.Balanced},
+				{"MODEL_DEEP", cfg.Model.Deep},
+			} {
+				if m.id == "" {
+					t.Errorf("%s resolved empty", m.key)
+				}
+			}
+		})
+	}
+}
+
+// parseDotenv reads the KEY=value lines an example file ships.
+//
+// Deliberately simple: it handles what these files actually contain — comments,
+// blank lines, trailing `# ...` notes on a value, and optional quotes. It is
+// not a general dotenv implementation, and an example needing one would be an
+// example too clever for the operator who has to read it.
+func parseDotenv(path string) (map[string]string, error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	env := map[string]string{}
+	scanner := bufio.NewScanner(f)
+	for scanner.Scan() {
+		line := strings.TrimSpace(scanner.Text())
+		if line == "" || strings.HasPrefix(line, "#") {
+			continue
+		}
+		line = strings.TrimPrefix(line, "export ")
+		key, value, ok := strings.Cut(line, "=")
+		if !ok {
+			continue
+		}
+		key = strings.TrimSpace(key)
+		// A trailing comment, but only when it is spaced off the value — a
+		// bare # inside a password is part of the password.
+		if i := strings.Index(value, " #"); i >= 0 {
+			value = value[:i]
+		}
+		value = strings.TrimSpace(value)
+		value = strings.Trim(value, `"'`)
+		env[key] = value
+	}
+	return env, scanner.Err()
+}

infrastructure/.env.docker.example

@@ -79,9 +79,48 @@ DATABASE_CONN_MAX_LIFETIME=30m
 DATABASE_CONNECT_TIMEOUT=10s
 DATABASE_STATEMENT_TIMEOUT=10s
 
+# ── Model gateway ───────────────────────────────────────────────────────────
+# THIS BLOCK WAS MISSING and a deployment copying this file could not start:
+# APP_ENV=production with no MODEL_API_KEY is refused, because the alternative
+# is an API that accepts agent runs and fails every one of them at the gateway.
+#
+# One wire protocol: the openai chat-completions shape. Groq, Gemini,
+# OpenRouter, Together, vLLM and a local Ollama all serve it, so switching
+# vendors is a base URL and a model id, not a code change.
+#
+# Empty MODEL_PROVIDER means openai — the only implementation. Empty
+# MODEL_BASE_URL means Groq.
+MODEL_PROVIDER=openai
+MODEL_BASE_URL=https://api.groq.com/openai/v1
+
+# REQUIRED in production. There is no ANTHROPIC_API_KEY fallback: that variable
+# is now REFUSED at startup if it is set while this one is empty, because
+# silently authenticating to Groq with a key named for a vendor this service
+# cannot call is a lie the next operator has to unpick. Rename it here, and
+# replace the value — an Anthropic key boots fine and then fails every run with
+# 401, which the startup check cannot catch and only the model call can.
+MODEL_API_KEY=
+
+# Model ids must be ones MODEL_BASE_URL actually serves. A leftover claude-*
+# id is refused at startup by name and tier: nothing configured serves one, so
+# every run on that tier would 400 at the gateway.
+MODEL_FAST=llama-3.1-8b-instant
+MODEL_BALANCED=llama-3.3-70b-versatile
+MODEL_DEEP=llama-3.3-70b-versatile
+
+# Off. Most non-reasoning models — the llama ids above included — reject the
+# whole request rather than ignoring reasoning_effort. Turn it on only for a
+# model documented to take it.
+MODEL_REASONING_EFFORT=
+
 # ── HTTP timeouts ───────────────────────────────────────────────────────────
 HTTP_READ_TIMEOUT=15s
-HTTP_WRITE_TIMEOUT=30s
+# 180s, not 30s. internal/config REFUSES TO START when this is below the deep
+# tier's 2m agent deadline: the server would abort the response mid-run and the
+# caller would see 502 from the proxy in front, a gateway error for something no
+# gateway did. 30s shipped here for a long time and was the cause of exactly
+# that incident. Anything at or under 2m0s is a container that will not boot.
+HTTP_WRITE_TIMEOUT=180s
 HTTP_IDLE_TIMEOUT=60s
 HTTP_SHUTDOWN_TIMEOUT=10s
 

infrastructure/docker-compose.yml

@@ -121,7 +121,9 @@ services:
       HTTP_HOST: 0.0.0.0
       HTTP_PORT: "8080"
       HTTP_READ_TIMEOUT: ${HTTP_READ_TIMEOUT:-15s}
-      HTTP_WRITE_TIMEOUT: ${HTTP_WRITE_TIMEOUT:-30s}
+      # Must exceed the deep tier's 2m agent deadline or config refuses to
+      # start — see .env.docker.example. The old 30s default could not boot.
+      HTTP_WRITE_TIMEOUT: ${HTTP_WRITE_TIMEOUT:-180s}
       HTTP_IDLE_TIMEOUT: ${HTTP_IDLE_TIMEOUT:-60s}
       HTTP_SHUTDOWN_TIMEOUT: ${HTTP_SHUTDOWN_TIMEOUT:-10s}
       # Browser origins allowed to call this API. Empty means same-origin only,