Files
krow_backend/Makefile
Suriyakumarvijayanayagam c74fe7e074 Add an OpenAI-compatible gateway, so the model provider is a config value
The platform could only talk to one vendor. Moving off Claude — for cost, or
because a client asks for Gemini — meant a rewrite behind an interface that
already had exactly the right shape and one implementation.

`openai` is not only OpenAI. Groq, Gemini's compatibility endpoint, OpenRouter,
Together, vLLM and a local Ollama all serve the chat-completions shape, so one
implementation reaches all of them and the difference between them is a base
URL and three model ids. That is why this is one file and not a package per
vendor.

`routing.go` had the vendor baked into the routing table every provider has to
read: effort was `anthropic.OutputConfigEffort`. Nothing was wrong with that
while there was one implementation; it became wrong the moment there were two,
because the OpenAI path would have had to import the Anthropic SDK to learn how
hard to think. Effort is now the platform's own three-value vocabulary and each
implementation maps it onto whatever its API calls the same idea.

THE ACCOUNTING DIFFERS BETWEEN THE TWO WIRES, and getting it wrong would have
been invisible. OpenAI reports prompt_tokens INCLUSIVE of the cached prefix;
Anthropic reports input tokens EXCLUSIVE of it and carries the cache
separately. Usage.Total() adds all four fields, so copying both numbers across
verbatim bills the cached prefix twice — worst on long conversations, which is
exactly where I3's budget matters most. The run would still answer; it would
just hit BudgetExceeded early, for no visible reason. normalise() subtracts,
and there is a test named after it.

Streamed tool calls are keyed by their wire index, not appended in arrival
order. Providers interleave the fragments of parallel calls, so appending
splices one call's arguments onto another's — and the result is usually two
calls that are each valid JSON and both wrong, which means the tools run with
inputs the model never chose and nothing errors. Mutation-checked: ignoring the
index produces `{"day"{"week":"friday"}:"next"}` and the test catches it.

Three configuration mistakes are refused at startup rather than at runtime:

  - MODEL_BASE_URL without MODEL_PROVIDER=openai. The anthropic path has one
    endpoint and ignores the field, so this is a deployment that believes it
    switched providers and did not — every run still goes to Anthropic and is
    still billed there, with nothing in the logs to say so. Cost is the whole
    reason this change exists, and that is the one mistake that silently
    defeats it.
  - An unrecognised MODEL_PROVIDER, once at boot instead of once per run.
  - A production deployment with no credential — except against localhost,
    which needs none, and demanding one would make the free local path
    impossible to configure.

reasoning_effort is opt-in via MODEL_REASONING_EFFORT. Reasoning models accept
it; most others reject the entire request with a 400 rather than ignoring an
unknown key, so every deployment would have had to opt out instead.

`make eval-live` now reads the same environment the service does and logs which
provider answered, because a suite that cannot say which model produced a
result is a suite whose result cannot be compared with another run's. That is
the point of this change: §12 leaves model hosting open, and this makes the
decision cheap to reverse and possible to settle on evidence. Weigh the I7 case
heaviest — a cheaper model that follows the planted injection is a security
regression, not a saving.

Default behaviour is unchanged: MODEL_PROVIDER unset means anthropic, and
ANTHROPIC_API_KEY still works, so no existing deployment needs an edit.

NOT verified against a live provider — no credential was available on this
machine. Tested against a fake endpoint covering both paths, and the three
guarantees above are mutation-checked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
2026-09-01 11:47:53 +05:30

272 lines
11 KiB
Makefile

# ============================================================================
# Krow backend — developer tasks
#
# Migration commands are thin wrappers over the golang-migrate CLI. The
# migration FILES are the source of truth for the schema; nothing here or in
# go-api/ ever issues DDL of its own.
#
# brew install golang-migrate # or see github.com/golang-migrate/migrate
# ============================================================================
SHELL := /bin/bash
.DEFAULT_GOAL := help
# Capture APP_ENV as the process environment supplied it, BEFORE .env is
# included. Make gives an included assignment precedence over the environment,
# so without this `APP_ENV=production make migrate-down` would silently read
# `development` out of .env and the guard below would not fire.
APP_ENV_FROM_ENVIRONMENT := $(APP_ENV)
ifneq (,$(wildcard .env))
include .env
export
endif
# The environment wins; .env is only the fallback. A command-line override
# (`make migrate-down APP_ENV=production`) outranks both, as Make intends.
ifneq (,$(APP_ENV_FROM_ENVIRONMENT))
APP_ENV := $(APP_ENV_FROM_ENVIRONMENT)
endif
MIGRATIONS_DIR ?= ./migrations
DATABASE_SCHEMA ?= public
DATABASE_SSLMODE ?= disable
# URL-encode the credentials and the database name. "Krow-force" is mixed-case
# and hyphenated, and a password can contain anything; neither is safe raw.
enc = $(shell printf '%s' '$(1)' | python3 -c 'import sys,urllib.parse; print(urllib.parse.quote(sys.stdin.read(), safe=""))')
DB_URL = postgres://$(call enc,$(DATABASE_USER)):$(call enc,$(DATABASE_PASSWORD))@$(DATABASE_HOST):$(DATABASE_PORT)/$(call enc,$(DATABASE_NAME))?sslmode=$(DATABASE_SSLMODE)&search_path=$(DATABASE_SCHEMA)
MIGRATE = migrate -path $(MIGRATIONS_DIR) -database "$(DB_URL)"
PSQL = psql -h $(DATABASE_HOST) -p $(DATABASE_PORT) -U $(DATABASE_USER) -d "$(DATABASE_NAME)"
# VERSION identifies a build. It defaults to the current commit (with -dirty if
# the tree has uncommitted changes), so a stamped build is the default rather
# than something to remember. Override for a release tag: VERSION=v1.2.0.
VERSION ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo dev)$(shell git diff --quiet 2>/dev/null || echo -dirty)
IMAGE ?= doormile/krowbackend:latest
# Exported, not just defined: docker-compose.yml reads ${VERSION} and ${IMAGE}
# from the environment, and a make variable is not in a recipe's environment
# unless it is exported. Without this the compose build arg falls back to "dev"
# and every image reports the same version.
export VERSION
export IMAGE
.PHONY: help
help: ## Show this help
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-18s\033[0m %s\n", $$1, $$2}'
# ── Go ──────────────────────────────────────────────────────────────────────
.PHONY: run
run: ## Run the API against the local database
cd go-api && go run ./cmd/api
.PHONY: build
build: ## Compile the API to go-api/bin/api
cd go-api && go build -ldflags="-X main.version=$(VERSION)" -o bin/api ./cmd/api
.PHONY: tidy
tidy: ## go mod tidy
cd go-api && go mod tidy
.PHONY: fmt
fmt: ## gofmt the module
cd go-api && gofmt -w ./cmd ./internal
.PHONY: vet
vet: ## go vet the module
cd go-api && go vet ./...
.PHONY: test
test: ## Run the Go tests
cd go-api && go test ./...
.PHONY: eval
eval: ## Run the agent eval suites (§9 — run this on every loop, retrieval or prompt change)
cd go-api && go test ./internal/evals/... -run "Suite|Detector" -v -count=1
.PHONY: import-agents
import-agents: ## Publish the specs in agents/ into a tenant: make import-agents ORG=<slug>
@test -n "$(ORG)" || { echo "usage: make import-agents ORG=<slug>"; exit 1; }
cd go-api && go run ./cmd/importagents --dir ../agents --skills ../skills --org "$(ORG)"
.PHONY: check-agents
check-agents: ## Parse every spec in agents/ and report, writing nothing
cd go-api && go run ./cmd/importagents --dir ../agents --skills ../skills --org check --dry-run
.PHONY: eval-live
eval-live: ## Run the eval suites against the REAL model (needs a key, costs tokens)
@test -n "$$MODEL_API_KEY" -o -n "$$ANTHROPIC_API_KEY" || { \
echo "eval-live needs MODEL_API_KEY (or ANTHROPIC_API_KEY) — it calls a real model and costs tokens."; \
echo "The scripted suites (make eval) are the gate; this is the confirmation."; \
echo ""; \
echo "To evaluate a different provider, point it somewhere else:"; \
echo " MODEL_PROVIDER=openai \\"; \
echo " MODEL_BASE_URL=https://api.groq.com/openai/v1 \\"; \
echo " MODEL_API_KEY=... MODEL_BALANCED=<model-id> make eval-live"; \
exit 1; }
cd go-api && go test ./internal/evals/ -run "TestLive" -v -count=1 -timeout 10m
.PHONY: ingest
ingest: ## Ingest knowledge/ into a tenant: make ingest ORG=<slug>
@test -n "$(ORG)" || { echo "usage: make ingest ORG=<slug>"; exit 1; }
cd go-api && go run ./cmd/ingest --dir ../knowledge --org "$(ORG)"
.PHONY: reembed
reembed: ## Re-embed a tenant's corpus with the current model: make reembed ORG=<slug>
@test -n "$(ORG)" || { echo "usage: make reembed ORG=<slug>"; exit 1; }
cd go-api && go run ./cmd/reembed --org "$(ORG)"
.PHONY: knowledge
knowledge: ## Run the retrieval layer's permission tests
cd go-api && go test ./internal/knowledge/ -v -count=1
.PHONY: tools
tools: ## Show what every agent tool returns against the seeded data
cd go-api && go test ./internal/tools/ -run TestDumpToolOutput -v
.PHONY: check
check: fmt vet test ## Format, vet and test
# ── Database ────────────────────────────────────────────────────────────────
.PHONY: db-create
db-create: ## Create the database if it does not exist (never drops anything)
@if psql -h $(DATABASE_HOST) -p $(DATABASE_PORT) -U $(DATABASE_USER) -d postgres -tAc \
"SELECT 1 FROM pg_database WHERE datname = '$(DATABASE_NAME)'" | grep -q 1; then \
echo "database \"$(DATABASE_NAME)\" already exists — nothing to do"; \
else \
psql -h $(DATABASE_HOST) -p $(DATABASE_PORT) -U $(DATABASE_USER) -d postgres -c \
'CREATE DATABASE "$(DATABASE_NAME)"' && echo "created \"$(DATABASE_NAME)\""; \
fi
.PHONY: seed
seed: ## Load the frontend demo dataset (idempotent upsert in one transaction)
cd go-api && SEED_FIXTURE_PATH="$(CURDIR)/seed/fixtures/seed.json" go run ./cmd/seed
# seed/fixtures/seed.json is GENERATED from krow-demo/src/api/seed.js. Edit the
# seed module, not the fixture. These two targets are the only supported way to
# change it; both need the frontend repo checked out beside this one.
.PHONY: seed-fixture
seed-fixture: ## Regenerate seed.json from the frontend seed module
cd "$(CURDIR)/../krow-demo" && npm run seed:fixture
.PHONY: seed-fixture-check
seed-fixture-check: ## Fail if seed.json no longer matches the frontend seed module
cd "$(CURDIR)/../krow-demo" && npm run seed:check
.PHONY: gen-resources
gen-resources: ## Regenerate domain descriptors from the live schema
python3 scripts/gen_resources.py > go-api/internal/domain/resources_gen.go
cd go-api && gofmt -w ./internal/domain
# Auth runs before routing, so an unauthenticated probe answers 401 for every
# path — including ones that do not exist. Verifying a deployment therefore
# needs a session, and the credentials come from the environment so they never
# reach argv or shell history:
#
# KROW_EMAIL=... KROW_PASSWORD=... make verify-deploy BASE=https://your-host
#
# Read-only. Add ARGS=--write to exercise the write paths as well.
.PHONY: verify-deploy
verify-deploy: ## Check every endpoint on a deployment: make verify-deploy BASE=<url>
@python3 scripts/verify-deploy.py "$(or $(BASE),http://127.0.0.1:8080)" $(ARGS)
.PHONY: db-health
db-health: ## Ask the running API for its database health
@curl -fsS http://$(HTTP_HOST):$(HTTP_PORT)/health | python3 -m json.tool
# ── Migrations ──────────────────────────────────────────────────────────────
.PHONY: migrate-up
migrate-up: ## Apply all pending migrations
$(MIGRATE) up
.PHONY: migrate-down
migrate-down: ## Roll back exactly one migration (guarded outside development)
@if [ "$(APP_ENV)" != "development" ]; then \
echo "refusing: migrate-down is only permitted when APP_ENV=development (got '$(APP_ENV)')"; \
echo "roll back staging/production deliberately, with a reviewed plan and a fresh backup."; \
exit 1; \
fi
$(MIGRATE) down 1
.PHONY: migrate-status
migrate-status: ## Show the currently applied migration version
@$(MIGRATE) version
.PHONY: migrate-new
migrate-new: ## Create a new migration pair: make migrate-new NAME=add_widgets
@test -n "$(NAME)" || { echo "usage: make migrate-new NAME=add_widgets"; exit 1; }
migrate create -ext sql -dir $(MIGRATIONS_DIR) -seq $(NAME)
.PHONY: migrate-force
migrate-force: ## Clear a dirty state: make migrate-force VERSION=1
@test -n "$(VERSION)" || { echo "usage: make migrate-force VERSION=1"; exit 1; }
$(MIGRATE) force $(VERSION)
# There is deliberately no `migrate-drop` target. golang-migrate's `drop`
# command deletes every table in the database; it must never be one keystroke
# away from a developer who meant `down`. Run it by hand if you truly need it.
.PHONY: verify-schema
verify-schema: ## Print the tables, enums, FKs and indexes that exist in the target schema
@$(PSQL) -f scripts/verify_schema.sql
# ── Docker ──────────────────────────────────────────────────────────────────
#
# These read infrastructure/.env, NOT the repository-root .env that `make run`
# uses. Two files on purpose: the root one points at a developer's database
# from the host, the infrastructure one points at a deployment's database from
# inside a container, and DATABASE_HOST is different in each.
COMPOSE_DIR := infrastructure
COMPOSE_FILES := -f docker-compose.yml
# Layer the local database in with: make docker-up LOCAL_DB=1
ifdef LOCAL_DB
COMPOSE_FILES += -f docker-compose.local-db.yml
endif
COMPOSE = cd $(COMPOSE_DIR) && docker compose $(COMPOSE_FILES)
.PHONY: docker-build
docker-build: ## Build the API image
$(COMPOSE) build
.PHONY: docker-up
docker-up: ## Start the stack (migrations run first). LOCAL_DB=1 adds PostgreSQL
$(COMPOSE) up -d --build
@echo "api: http://127.0.0.1:8080/health"
@echo "logs: make docker-logs"
.PHONY: docker-down
docker-down: ## Stop the stack, keeping volumes
$(COMPOSE) down
.PHONY: docker-logs
docker-logs: ## Follow the API logs
$(COMPOSE) logs -f api
.PHONY: docker-ps
docker-ps: ## Show stack status and health
$(COMPOSE) ps
.PHONY: docker-migrate
docker-migrate: ## Apply migrations only, without starting the API
$(COMPOSE) run --rm migrate
.PHONY: docker-seed
docker-seed: ## Load the demo fixture into the deployment's database
$(COMPOSE) run --rm --entrypoint /usr/local/bin/seed api
.PHONY: docker-setpassword
docker-setpassword: ## Set a password: make docker-setpassword EMAIL=demo@krow.app
@test -n "$(EMAIL)" || { echo "usage: make docker-setpassword EMAIL=demo@krow.app"; exit 1; }
$(COMPOSE) run --rm -it --entrypoint /usr/local/bin/setpassword api -email $(EMAIL)
.PHONY: docker-config
docker-config: ## Render the fully resolved compose configuration
$(COMPOSE) config