backend_fiesta/docs/SCAN_TO_ORDER.md

Scan-to-order — mobile integration

A customer photographs a product. Google Lens (on the phone) turns the photo into a label — "Milk Bikis", "Dabur Honey 500g". The app sends that label here and gets back: what the product is, which of the customer's stores sell it, in which sizes, with live stock, nearest first, and which store we recommend. When the customer taps a store and a size, a second call confirms the shelf still has it — and if it does not, names the next-nearest store that does.

Base path: /live/api/v1/mob/scan. Every response uses the usual envelope { code, status, message, details }; the shapes below are details.

The flow

photo ──Lens──▶ label
                 │
                 ▼
   POST /lookup  ───▶  match + stores[] (recommended first)
                 │
   customer taps a store + a size
                 │
                 ▼
   POST /confirm ───▶  ok:true            → add to basket with existing order APIs
                       ok:false + alternative → offer the other store

GET /stores is for the "choose another shop" sheet: the customer's registered stores, nearest first, independent of any product.

POST /lookup

{
  "customerid": 5123,
  "label": "Milk Bikis",
  "latitude": 11.0290,          // phone fix; optional — saved address is used without it
  "longitude": 77.0290,
  "tenantids": [1135, 1140],    // optional: what the app THINKS the customer joined
  "limit": 0                    // optional: max stores, 0 = all
}

tenantids is verified, never trusted: the server intersects it with the tenantcustomers table. Ids the customer is not actually registered with come back in unregistered_tenantids — treat that as "refresh the local list". A list that matches nothing at all is treated as stale and all registered stores are used.

Response:

{
  "label": "Milk Bikis",
  "match": {
    "brand": "britannia", "catalogueid": 7, "imageid": "britannia_milk_bikis_100g",
    "product_name": "Milk Bikis", "size": "100 g", "variant_key": "milk_bikis",
    "image": "https://…", "score": 0.94, "method": "vector+text"
  },
  "catalogue_variants": [ { "…same shape…": "100 g" }, { "…": "200 g" } ],
  "confidence": 0.94,
  "available": true,
  "recommended_locationid": 20,
  "stores": [
    {
      "tenantid": 2, "tenantname": "R Mart", "locationid": 20, "locationname": "Hopes",
      "latitude": 11.01, "longitude": 77.0, "distance_km": 3.8, "open": true,
      "deliveryradius": 5, "deliverymins": 30,
      "recommended": true, "available": true,
      "options": [
        { "productid": 200, "productname": "Milk Bikis 100g", "size": "100 g", "price": 12, "stock": 6,
          "available": true, "is_variant": false, "matched_by": "imageid", "image": "…" },
        { "productid": 201, "productname": "Milk Bikis 200g", "size": "200 g", "price": 22, "stock": 3,
          "available": true, "is_variant": true, "variantname": "200 g", "matched_by": "variant-of:200" }
      ]
    },
    { "locationid": 10, "locationname": "Peelamedu", "distance_km": 0.9, "available": false, "recommended": false,
      "options": [ { "productid": 100, "stock": 0, "available": false, "…": "…" } ] }
  ],
  "unregistered_tenantids": [],
  "message": "Available at 1 of your stores."
}

How to read it:

• match == null → nothing recognised; show message and let them retry. confidence below ~0.5 → recognised but unsure; confirm the name with the customer before showing prices. method: "text" means no embedding model was involved (not configured, or it timed out) — be a little more cautious.
• stores is ordered in-stock first, then nearest. Exactly one store has recommended: true — the nearest with stock — and only when available is true. Stores that sell it but have nothing on the shelf are still listed (so the customer understands why they are not recommended); stores that do not sell it are not.
• options are the things that can actually go in a basket at that store — the matched product and each of its sizes — each a real product with its own productid, price and live stock. Use productid in the existing cart/order calls exactly as you would from the catalogue screen.
• distance_km: -1 means the distance is unknown (no fix from the phone and no saved address, or the store has no coordinates). Do not render it as 0. Send latitude/longitude on /confirm too if you display distance from its reply: the saved address is only consulted there when the shelf is empty and alternatives have to be ranked, so without a fix the store you tapped comes back -1.

POST /confirm

Sent when the customer taps a store and an option. Re-reads live stock — nothing is cached on this path.

{ "customerid": 5123, "tenantid": 1, "locationid": 10, "productid": 100, "quantity": 2,
  "latitude": 11.029, "longitude": 77.029 }

{
  "ok": false,
  "reason": "out_of_stock",           // in_stock | insufficient_stock | out_of_stock | not_sold_here | store_not_registered
  "store":  { "…the store they tapped…" },   // distance_km filled from the fix you send
  "option": { "productid": 100, "stock": 0, "…": "…" },
  "requested": 2,
  "alternative": {                     // absent when nobody has enough
    "locationid": 20, "locationname": "Hopes", "distance_km": 3.8, "recommended": true, "available": true,
    "options": [ { "productid": 200, "stock": 6, "price": 12, "…": "…" } ]
  },
  "message": "Out of stock at Peelamedu. Hopes has it (3.8 km away)."
}

ok: true → proceed to the basket. ok: false → show message; if alternative is present offer it as a one-tap switch (it is the same product, not another size — the customer chose a size and we do not substitute). These are HTTP 200s: they are answers, not errors.

GET /stores?customerid=5123&latitude=11.029&longitude=77.029

The customer's registered stores, nearest first, distance_km: -1 last. Same ScanStore shape as inside stores[] above, without options.

Errors (HTTP status ≠ 200)

Status	When
400	Missing customerid/label/ids, or a body that is not JSON. message says which.
404	customerid does not exist.
503	The catalogue database is not reachable. Retry later; the rest of the app is unaffected.
500	Anything else. Logged server-side.

Behind the curtain (for whoever operates it)

• Recognition = pgvector cosine search over every brand_* table in the catalogue (each with its own index, merged), plus a word match on product_name/title/search_query that settles near-ties and works on its own when no embedding model is configured. The model is set by EMBEDDING_PROVIDER/MODEL/API_KEY and must be the one that indexed the catalogue — the first search checks the vector width and refuses a mismatch by name.
• The word match asks for most of the label, not all of it (minTokenHits: two thirds, rounded up, and both of a two-word label). Requiring every word meant one word the catalogue does not use took the right product out of the running entirely — "Dettol bottle pack" retrieved no Dettol, "Parle G biscuit pack" retrieved no Parle-G — and the vector search then answered alone, confidently and wrongly, at a score the floor could not catch. Each brand's rows are ordered by how much of the label they carry (the whole label as a substring outranks any number of loose words) so that the per-brand LIMIT keeps the best rows and not merely the first ones the planner reached. Packaging words — "pack", "bottle", "jar", "sachet" and friends, see utils.isPackaging — are dropped before any of this, like pack sizes, unless the label is nothing else.
• The catalogue's model (verified 2026-09-15 by cosine against a stored row: 1.0000): all-MiniLM-L6-v2, 384-d, unit-normalised, embedding the search_query column (brand + name + category + blurb + price range). Ollama ships it as all-minilm; the cluster's ollama.krow service serves it, so production is:

  EMBEDDING_PROVIDER=openai
  EMBEDDING_BASE_URL=http://ollama.krow.svc.cluster.local:11434/v1
  EMBEDDING_MODEL=all-minilm
  EMBEDDING_API_KEY=ollama        # any non-empty value; Ollama ignores it
  EMBEDDING_DIMENSIONS=384
  

  A bare label ("Milk Bikis") scores ~0.92 against its product's stored vector and ~0.23 against an unrelated one, which is what the 0.50 floor in scanService.go is set against — the middle of that split, not the edge of the noise. It was 0.30 until a near-miss got through in production ("Paracetamol" → "Paneer Makhni 500ml", 0.304). If the catalogue team ever re-embeds with another model, change EMBEDDING_MODEL/DIMENSIONS here and nothing else.
• Speed: the label's vector (7 days) and the ranked catalogue hits (30 min) are cached in Redis and in-process, so a popular product costs one model call platform-wide. Customer, stores and catalogue are read concurrently; the whole lookup is capped at 5 s and a slow model degrades to a text answer instead of a spinner. Live stock is one indexed query and is never cached.
• Availability is the same rule the app's catalogue screen uses: products.approve = 1, productlocations.publishedat IS NOT NULL, stock = live SUM(in) − SUM(out) of productstocks at that outlet, price = the outlet's own price else the tenant's retail price.
• No reservation. Confirm re-reads the ledger; a hold would give the same answer with a timer to babysit. If contention becomes real, a Redis-backed short hold slots in at Confirm without changing the API.
• Identity is the customerid in the body, like every other mobile endpoint here — there is no auth layer yet (see SECURITY_HANDOFF.md).

For backend developers

Where the code is

File	Holds
models/scan.go	request/response shapes (ScanLookupRequest, ScanStoreOffer, ScanOption, …)
repositories/scanRepository.go	all SQL: registered stores, live options, vector + text search, the two-tier cache
services/scanService.go	the pipeline: parallel reads, scoring, family grouping, ranking, confirm fallback
controllers/scanController.go	the three handlers and the error → status mapping
routes/scanroutes.go	/v1/mob/scan/*
utils/embedding.go	Embedder interface, OpenAI-compatible and Gemini clients
utils/geo.go	coordinate parsing, haversine, opening hours, label tokenising
config/config.go	EmbeddingConfig and its validation
scratch/cataloguedims	read-only check of the catalogue's embedding width / fill

Try it locally

go run .    # with the local compose stack; EMBEDDING_* unset → text-only, still works
curl -s localhost:1122/live/api/v1/mob/scan/lookup -H 'Content-Type: application/json' \
  -d '{"customerid":1,"label":"Milk Bikis","latitude":11.03,"longitude":77.03}' | jq .details

To exercise the vector path locally, run Ollama on your Mac (ollama pull all-minilm) and set EMBEDDING_PROVIDER=openai, EMBEDDING_BASE_URL=http://localhost:11434/v1, EMBEDDING_MODEL=all-minilm, EMBEDDING_API_KEY=ollama, EMBEDDING_DIMENSIONS=384 in .env.local. The local catalogue must carry vectors from the same model for results to mean anything; a schema-only dump does not.

Tests

go test ./services -run 'Lookup|Confirm|Stores|CatalogueFamily' drives the whole pipeline through a fake repository (services/scan_test.go); no database. go test ./utils covers both HTTP clients against httptest servers, and the geo helpers. Add a case to scan_test.go's fixture when you change ranking — it is the spec.

Knobs (constants in scanService.go)

Constant	Default	Effect
scanLookupTimeout	5 s	whole lookup, including the model call
scanCatalogueTopK	15	rows taken from each brand table and from the merge
scanMinScore	0.50	below this the best hit is not shown as a match
embedTimeout (utils/embedding.go)	4 s	one model call
scanVectorTTL / scanHitsTTL (scanRepository.go)	7 d / 30 min	cache lifetimes

Scores: vector = 1 − cosine distance; text = 0.95 for the whole label inside the name, else 0.8 × (label words found / label words); combined = max(vector, text) + 0.10 when both hit, capped at 1. Ties are broken by cosine distance — nearest first, a text-only row last — and only then by name.

The label and the product name are both separator-folded before that substring test (utils.FoldSeparators), and compared again with separators removed (utils.TightenLabel, labels of 4+ characters), so the brand's own punctuation does not decide the match: "Parle G", "Parle-G" and "ParleG" all reach Parle-G Original Glucose Biscuits. A single-character token survives tokenising when it follows a word, because it is often the whole name — the "G" of Parle-G, the "K" of Special K. It is still dropped when it stands alone or is a pack multiplier.

All three mattered at once: before this, "Parle G" tied with Parle Monaco Classic at 0.9 (the "G" was dropped, so only "parle" matched either row), and the name tie-break handed it to Monaco because a space precedes a hyphen in ASCII. A confident, wrong answer — the kind no score floor can catch.

Changing the embedding model

1. The catalogue team re-embeds search_query with the new model.
2. Serve it (Ollama pull, or a hosted key).
3. Change EMBEDDING_MODEL / EMBEDDING_DIMENSIONS (and provider/URL if needed) in the cluster; roll the pods.
4. Flush the hit cache if you cannot wait 30 min: keys are scan:hits:v1:* and scan:emb:v1:* in Redis (they are also keyed by model name, so old entries simply stop being read).

Nothing in Go changes. A width mismatch fails the first search with an error naming both numbers.

Adding a provider

Implement utils.Embedder (Embed(ctx, text) ([]float32, error) and Model() string), add a case to NewEmbedder, and add the provider name to the allow-list in config.validate. Keep the HTTP client timeout: the customer is holding a phone.