README: a map of the service for new developers

What it is, how to run it, how configuration works, the module layout,
the seven steps to add an endpoint, the standing surprises, and a
Kubernetes cheat-sheet — each pointing at the detailed doc. Plus a
backend-developer section in SCAN_TO_ORDER.md: file map, local try-out,
tests, tuning knobs, and how to change the embedding model or add a
provider.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-15 17:06:42 +05:30
parent 369e9fc7b4
commit aaea1bfc00
2 changed files with 211 additions and 0 deletions

View File

@@ -186,3 +186,76 @@ Same `ScanStore` shape as inside `stores[]` above, without options.
Redis-backed short hold slots in at `Confirm` without changing the API.
- **Identity** is the `customerid` in the body, like every other mobile
endpoint here — there is no auth layer yet (see `SECURITY_HANDOFF.md`).
## For backend developers
### Where the code is
| File | Holds |
|---|---|
| `models/scan.go` | request/response shapes (`ScanLookupRequest`, `ScanStoreOffer`, `ScanOption`, …) |
| `repositories/scanRepository.go` | all SQL: registered stores, live options, vector + text search, the two-tier cache |
| `services/scanService.go` | the pipeline: parallel reads, scoring, family grouping, ranking, confirm fallback |
| `controllers/scanController.go` | the three handlers and the error → status mapping |
| `routes/scanroutes.go` | `/v1/mob/scan/*` |
| `utils/embedding.go` | `Embedder` interface, OpenAI-compatible and Gemini clients |
| `utils/geo.go` | coordinate parsing, haversine, opening hours, label tokenising |
| `config/config.go` | `EmbeddingConfig` and its validation |
| `scratch/cataloguedims` | read-only check of the catalogue's embedding width / fill |
### Try it locally
```sh
go run . # with the local compose stack; EMBEDDING_* unset → text-only, still works
curl -s localhost:1122/live/api/v1/mob/scan/lookup -H 'Content-Type: application/json' \
-d '{"customerid":1,"label":"Milk Bikis","latitude":11.03,"longitude":77.03}' | jq .details
```
To exercise the vector path locally, run Ollama on your Mac
(`ollama pull all-minilm`) and set `EMBEDDING_PROVIDER=openai`,
`EMBEDDING_BASE_URL=http://localhost:11434/v1`, `EMBEDDING_MODEL=all-minilm`,
`EMBEDDING_API_KEY=ollama`, `EMBEDDING_DIMENSIONS=384` in `.env.local`. The
local catalogue must carry vectors from the same model for results to mean
anything; a schema-only dump does not.
### Tests
`go test ./services -run 'Lookup|Confirm|Stores|CatalogueFamily'` drives
the whole pipeline through a fake repository (`services/scan_test.go`); no
database. `go test ./utils` covers both HTTP clients against `httptest`
servers, and the geo helpers. Add a case to `scan_test.go`'s fixture when
you change ranking — it is the spec.
### Knobs (constants in `scanService.go`)
| Constant | Default | Effect |
|---|---|---|
| `scanLookupTimeout` | 5 s | whole lookup, including the model call |
| `scanCatalogueTopK` | 15 | rows taken from each brand table and from the merge |
| `scanMinScore` | 0.30 | below this the best hit is not shown as a match |
| `embedTimeout` (`utils/embedding.go`) | 4 s | one model call |
| `scanVectorTTL` / `scanHitsTTL` (`scanRepository.go`) | 7 d / 30 min | cache lifetimes |
Scores: vector = `1 − cosine distance`; text = 0.95 for the whole label
inside the name, else `0.8 × (label words found / label words)`; combined =
`max(vector, text) + 0.10` when both hit, capped at 1.
### Changing the embedding model
1. The catalogue team re-embeds `search_query` with the new model.
2. Serve it (Ollama pull, or a hosted key).
3. Change `EMBEDDING_MODEL` / `EMBEDDING_DIMENSIONS` (and provider/URL if
needed) in the cluster; roll the pods.
4. Flush the hit cache if you cannot wait 30 min: keys are
`scan:hits:v1:*` and `scan:emb:v1:*` in Redis (they are also keyed by
model name, so old entries simply stop being read).
Nothing in Go changes. A width mismatch fails the first search with an error
naming both numbers.
### Adding a provider
Implement `utils.Embedder` (`Embed(ctx, text) ([]float32, error)` and
`Model() string`), add a case to `NewEmbedder`, and add the provider name to
the allow-list in `config.validate`. Keep the HTTP client timeout: the
customer is holding a phone.