README: a map of the service for new developers
What it is, how to run it, how configuration works, the module layout, the seven steps to add an endpoint, the standing surprises, and a Kubernetes cheat-sheet — each pointing at the detailed doc. Plus a backend-developer section in SCAN_TO_ORDER.md: file map, local try-out, tests, tuning knobs, and how to change the embedding model or add a provider. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -186,3 +186,76 @@ Same `ScanStore` shape as inside `stores[]` above, without options.
|
||||
Redis-backed short hold slots in at `Confirm` without changing the API.
|
||||
- **Identity** is the `customerid` in the body, like every other mobile
|
||||
endpoint here — there is no auth layer yet (see `SECURITY_HANDOFF.md`).
|
||||
|
||||
## For backend developers
|
||||
|
||||
### Where the code is
|
||||
|
||||
| File | Holds |
|
||||
|---|---|
|
||||
| `models/scan.go` | request/response shapes (`ScanLookupRequest`, `ScanStoreOffer`, `ScanOption`, …) |
|
||||
| `repositories/scanRepository.go` | all SQL: registered stores, live options, vector + text search, the two-tier cache |
|
||||
| `services/scanService.go` | the pipeline: parallel reads, scoring, family grouping, ranking, confirm fallback |
|
||||
| `controllers/scanController.go` | the three handlers and the error → status mapping |
|
||||
| `routes/scanroutes.go` | `/v1/mob/scan/*` |
|
||||
| `utils/embedding.go` | `Embedder` interface, OpenAI-compatible and Gemini clients |
|
||||
| `utils/geo.go` | coordinate parsing, haversine, opening hours, label tokenising |
|
||||
| `config/config.go` | `EmbeddingConfig` and its validation |
|
||||
| `scratch/cataloguedims` | read-only check of the catalogue's embedding width / fill |
|
||||
|
||||
### Try it locally
|
||||
|
||||
```sh
|
||||
go run . # with the local compose stack; EMBEDDING_* unset → text-only, still works
|
||||
curl -s localhost:1122/live/api/v1/mob/scan/lookup -H 'Content-Type: application/json' \
|
||||
-d '{"customerid":1,"label":"Milk Bikis","latitude":11.03,"longitude":77.03}' | jq .details
|
||||
```
|
||||
|
||||
To exercise the vector path locally, run Ollama on your Mac
|
||||
(`ollama pull all-minilm`) and set `EMBEDDING_PROVIDER=openai`,
|
||||
`EMBEDDING_BASE_URL=http://localhost:11434/v1`, `EMBEDDING_MODEL=all-minilm`,
|
||||
`EMBEDDING_API_KEY=ollama`, `EMBEDDING_DIMENSIONS=384` in `.env.local`. The
|
||||
local catalogue must carry vectors from the same model for results to mean
|
||||
anything; a schema-only dump does not.
|
||||
|
||||
### Tests
|
||||
|
||||
`go test ./services -run 'Lookup|Confirm|Stores|CatalogueFamily'` drives
|
||||
the whole pipeline through a fake repository (`services/scan_test.go`); no
|
||||
database. `go test ./utils` covers both HTTP clients against `httptest`
|
||||
servers, and the geo helpers. Add a case to `scan_test.go`'s fixture when
|
||||
you change ranking — it is the spec.
|
||||
|
||||
### Knobs (constants in `scanService.go`)
|
||||
|
||||
| Constant | Default | Effect |
|
||||
|---|---|---|
|
||||
| `scanLookupTimeout` | 5 s | whole lookup, including the model call |
|
||||
| `scanCatalogueTopK` | 15 | rows taken from each brand table and from the merge |
|
||||
| `scanMinScore` | 0.30 | below this the best hit is not shown as a match |
|
||||
| `embedTimeout` (`utils/embedding.go`) | 4 s | one model call |
|
||||
| `scanVectorTTL` / `scanHitsTTL` (`scanRepository.go`) | 7 d / 30 min | cache lifetimes |
|
||||
|
||||
Scores: vector = `1 − cosine distance`; text = 0.95 for the whole label
|
||||
inside the name, else `0.8 × (label words found / label words)`; combined =
|
||||
`max(vector, text) + 0.10` when both hit, capped at 1.
|
||||
|
||||
### Changing the embedding model
|
||||
|
||||
1. The catalogue team re-embeds `search_query` with the new model.
|
||||
2. Serve it (Ollama pull, or a hosted key).
|
||||
3. Change `EMBEDDING_MODEL` / `EMBEDDING_DIMENSIONS` (and provider/URL if
|
||||
needed) in the cluster; roll the pods.
|
||||
4. Flush the hit cache if you cannot wait 30 min: keys are
|
||||
`scan:hits:v1:*` and `scan:emb:v1:*` in Redis (they are also keyed by
|
||||
model name, so old entries simply stop being read).
|
||||
|
||||
Nothing in Go changes. A width mismatch fails the first search with an error
|
||||
naming both numbers.
|
||||
|
||||
### Adding a provider
|
||||
|
||||
Implement `utils.Embedder` (`Embed(ctx, text) ([]float32, error)` and
|
||||
`Model() string`), add a case to `NewEmbedder`, and add the provider name to
|
||||
the allow-list in `config.validate`. Keep the HTTP client timeout: the
|
||||
customer is holding a phone.
|
||||
|
||||
Reference in New Issue
Block a user