Files
backend_fiesta/docs/SCAN_TO_ORDER.md
Suriyakumarvijayanayagam 72907dae74 Scan-to-order: label from the customer's camera to "buy it here"
POST /v1/mob/scan/lookup   label + customer → catalogue match, sizes, and
                           every registered store that sells it with live
                           stock, in-stock first / nearest first, one
                           recommended
POST /v1/mob/scan/confirm  chosen store + size + qty → re-read the ledger;
                           ok, or the next-nearest store with enough of the
                           same product
GET  /v1/mob/scan/stores   registered stores nearest first

Recognition is pgvector cosine search over every brand_* table (each
with its own index, merged) plus a word match that settles near-ties
and works alone when no model is configured. The embedder is chosen by
EMBEDDING_PROVIDER (OpenAI-compatible or Gemini) and must be the model
that indexed the catalogue: verified 2026-09-15 as all-MiniLM-L6-v2 over
search_query, served by the cluster's Ollama as `all-minilm`; the first
search refuses a width mismatch by name.

Customer, stores and catalogue are read concurrently under a 5 s cap; a
slow model degrades to a text answer. Vectors and ranked hits are cached
in Redis and in-process; live stock never is. Availability uses the same
rules as the customer catalogue (approve, publishedat, ledger balance,
outlet price else retail). No stock reservation: confirm re-reads.

scratch/cataloguedims reports the catalogue's embedding width and fill.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 17:04:34 +05:30

189 lines
8.4 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Scan-to-order — mobile integration
A customer photographs a product. Google Lens (on the phone) turns the photo
into a label — `"Milk Bikis"`, `"Dabur Honey 500g"`. The app sends that label
here and gets back: what the product is, which of the customer's stores sell
it, in which sizes, with live stock, nearest first, and which store we
recommend. When the customer taps a store and a size, a second call confirms
the shelf still has it — and if it does not, names the next-nearest store
that does.
Base path: `/live/api/v1/mob/scan`. Every response uses the usual envelope
`{ code, status, message, details }`; the shapes below are `details`.
## The flow
```
photo ──Lens──▶ label
│
▼
POST /lookup ───▶ match + stores[] (recommended first)
│
customer taps a store + a size
│
▼
POST /confirm ───▶ ok:true → add to basket with existing order APIs
ok:false + alternative → offer the other store
```
`GET /stores` is for the "choose another shop" sheet: the customer's
registered stores, nearest first, independent of any product.
## `POST /lookup`
```json
{
"customerid": 5123,
"label": "Milk Bikis",
"latitude": 11.0290, // phone fix; optional — saved address is used without it
"longitude": 77.0290,
"tenantids": [1135, 1140], // optional: what the app THINKS the customer joined
"limit": 0 // optional: max stores, 0 = all
}
```
`tenantids` is verified, never trusted: the server intersects it with the
`tenantcustomers` table. Ids the customer is not actually registered with
come back in `unregistered_tenantids` — treat that as "refresh the local
list". A list that matches nothing at all is treated as stale and all
registered stores are used.
Response:
```json
{
"label": "Milk Bikis",
"match": {
"brand": "britannia", "catalogueid": 7, "imageid": "britannia_milk_bikis_100g",
"product_name": "Milk Bikis", "size": "100 g", "variant_key": "milk_bikis",
"image": "https://…", "score": 0.94, "method": "vector+text"
},
"catalogue_variants": [ { "…same shape…": "100 g" }, { "…": "200 g" } ],
"confidence": 0.94,
"available": true,
"recommended_locationid": 20,
"stores": [
{
"tenantid": 2, "tenantname": "R Mart", "locationid": 20, "locationname": "Hopes",
"latitude": 11.01, "longitude": 77.0, "distance_km": 3.8, "open": true,
"deliveryradius": 5, "deliverymins": 30,
"recommended": true, "available": true,
"options": [
{ "productid": 200, "productname": "Milk Bikis 100g", "size": "100 g", "price": 12, "stock": 6,
"available": true, "is_variant": false, "matched_by": "imageid", "image": "…" },
{ "productid": 201, "productname": "Milk Bikis 200g", "size": "200 g", "price": 22, "stock": 3,
"available": true, "is_variant": true, "variantname": "200 g", "matched_by": "variant-of:200" }
]
},
{ "locationid": 10, "locationname": "Peelamedu", "distance_km": 0.9, "available": false, "recommended": false,
"options": [ { "productid": 100, "stock": 0, "available": false, "…": "…" } ] }
],
"unregistered_tenantids": [],
"message": "Available at 1 of your stores."
}
```
How to read it:
- `match == null` → nothing recognised; show `message` and let them retry.
`confidence` below ~0.5 → recognised but unsure; confirm the name with the
customer before showing prices. `method: "text"` means no embedding model
was involved (not configured, or it timed out) — be a little more cautious.
- `stores` is ordered **in-stock first, then nearest**. Exactly one store has
`recommended: true` — the nearest with stock — and only when `available`
is true. Stores that sell it but have nothing on the shelf are still listed
(so the customer understands why they are not recommended); stores that do
not sell it are not.
- `options` are the things that can actually go in a basket at that store —
the matched product and each of its sizes — each a real product with its
own `productid`, price and live `stock`. Use `productid` in the existing
cart/order calls exactly as you would from the catalogue screen.
- `distance_km: -1` means the distance is unknown (no fix from the phone and
no saved address, or the store has no coordinates). Do not render it as 0.
## `POST /confirm`
Sent when the customer taps a store and an option. Re-reads live stock —
nothing is cached on this path.
```json
{ "customerid": 5123, "tenantid": 1, "locationid": 10, "productid": 100, "quantity": 2,
"latitude": 11.029, "longitude": 77.029 }
```
```json
{
"ok": false,
"reason": "out_of_stock", // in_stock | insufficient_stock | out_of_stock | not_sold_here | store_not_registered
"store": { "…the store they tapped…" },
"option": { "productid": 100, "stock": 0, "…": "…" },
"requested": 2,
"alternative": { // absent when nobody has enough
"locationid": 20, "locationname": "Hopes", "distance_km": 3.8, "recommended": true, "available": true,
"options": [ { "productid": 200, "stock": 6, "price": 12, "…": "…" } ]
},
"message": "Out of stock at Peelamedu. Hopes has it (3.8 km away)."
}
```
`ok: true` → proceed to the basket. `ok: false` → show `message`; if
`alternative` is present offer it as a one-tap switch (it is the **same
product**, not another size — the customer chose a size and we do not
substitute). These are HTTP 200s: they are answers, not errors.
## `GET /stores?customerid=5123&latitude=11.029&longitude=77.029`
The customer's registered stores, nearest first, `distance_km: -1` last.
Same `ScanStore` shape as inside `stores[]` above, without options.
## Errors (HTTP status ≠ 200)
| Status | When |
|---|---|
| 400 | Missing `customerid`/`label`/ids, or a body that is not JSON. `message` says which. |
| 404 | `customerid` does not exist. |
| 503 | The catalogue database is not reachable. Retry later; the rest of the app is unaffected. |
| 500 | Anything else. Logged server-side. |
## Behind the curtain (for whoever operates it)
- **Recognition** = pgvector cosine search over every `brand_*` table in the
catalogue (each with its own index, merged), plus a word match on
`product_name`/`title`/`search_query` that settles near-ties and works on
its own when no embedding model is configured. The model is set by
`EMBEDDING_PROVIDER/MODEL/API_KEY` and **must** be the one that indexed
the catalogue — the first search checks the vector width and refuses a
mismatch by name.
- **The catalogue's model** (verified 2026-09-15 by cosine against a stored
row: 1.0000): `all-MiniLM-L6-v2`, 384-d, unit-normalised, embedding the
`search_query` column (brand + name + category + blurb + price range).
Ollama ships it as `all-minilm`; the cluster's `ollama.krow` service serves
it, so production is:
```
EMBEDDING_PROVIDER=openai
EMBEDDING_BASE_URL=http://ollama.krow.svc.cluster.local:11434/v1
EMBEDDING_MODEL=all-minilm
EMBEDDING_API_KEY=ollama # any non-empty value; Ollama ignores it
EMBEDDING_DIMENSIONS=384
```
A bare label ("Milk Bikis") scores ~0.92 against its product's stored
vector and ~0.23 against an unrelated one, which is what the 0.30 floor in
`scanService.go` is set against. If the catalogue team ever re-embeds
with another model, change `EMBEDDING_MODEL`/`DIMENSIONS` here and
nothing else.
- **Speed**: the label's vector (7 days) and the ranked catalogue hits
(30 min) are cached in Redis and in-process, so a popular product costs
one model call platform-wide. Customer, stores and catalogue are read
concurrently; the whole lookup is capped at 5 s and a slow model degrades
to a text answer instead of a spinner. Live stock is one indexed query and
is never cached.
- **Availability** is the same rule the app's catalogue screen uses:
`products.approve = 1`, `productlocations.publishedat IS NOT NULL`, stock =
live `SUM(in) − SUM(out)` of `productstocks` at that outlet, price = the
outlet's own price else the tenant's retail price.
- **No reservation.** Confirm re-reads the ledger; a hold would give the
same answer with a timer to babysit. If contention becomes real, a
Redis-backed short hold slots in at `Confirm` without changing the API.
- **Identity** is the `customerid` in the body, like every other mobile
endpoint here — there is no auth layer yet (see `SECURITY_HANDOFF.md`).