POST /v1/mob/scan/lookup label + customer → catalogue match, sizes, and
every registered store that sells it with live
stock, in-stock first / nearest first, one
recommended
POST /v1/mob/scan/confirm chosen store + size + qty → re-read the ledger;
ok, or the next-nearest store with enough of the
same product
GET /v1/mob/scan/stores registered stores nearest first
Recognition is pgvector cosine search over every brand_* table (each
with its own index, merged) plus a word match that settles near-ties
and works alone when no model is configured. The embedder is chosen by
EMBEDDING_PROVIDER (OpenAI-compatible or Gemini) and must be the model
that indexed the catalogue: verified 2026-09-15 as all-MiniLM-L6-v2 over
search_query, served by the cluster's Ollama as `all-minilm`; the first
search refuses a width mismatch by name.
Customer, stores and catalogue are read concurrently under a 5 s cap; a
slow model degrades to a text answer. Vectors and ranked hits are cached
in Redis and in-process; live stock never is. Availability uses the same
rules as the customer catalogue (approve, publishedat, ledger balance,
outlet price else retail). No stock reservation: confirm re-reads.
scratch/cataloguedims reports the catalogue's embedding width and fill.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
189 lines
8.4 KiB
Markdown
189 lines
8.4 KiB
Markdown
# Scan-to-order — mobile integration
|
||
|
||
A customer photographs a product. Google Lens (on the phone) turns the photo
|
||
into a label — `"Milk Bikis"`, `"Dabur Honey 500g"`. The app sends that label
|
||
here and gets back: what the product is, which of the customer's stores sell
|
||
it, in which sizes, with live stock, nearest first, and which store we
|
||
recommend. When the customer taps a store and a size, a second call confirms
|
||
the shelf still has it — and if it does not, names the next-nearest store
|
||
that does.
|
||
|
||
Base path: `/live/api/v1/mob/scan`. Every response uses the usual envelope
|
||
`{ code, status, message, details }`; the shapes below are `details`.
|
||
|
||
## The flow
|
||
|
||
```
|
||
photo ──Lens──▶ label
|
||
│
|
||
▼
|
||
POST /lookup ───▶ match + stores[] (recommended first)
|
||
│
|
||
customer taps a store + a size
|
||
│
|
||
▼
|
||
POST /confirm ───▶ ok:true → add to basket with existing order APIs
|
||
ok:false + alternative → offer the other store
|
||
```
|
||
|
||
`GET /stores` is for the "choose another shop" sheet: the customer's
|
||
registered stores, nearest first, independent of any product.
|
||
|
||
## `POST /lookup`
|
||
|
||
```json
|
||
{
|
||
"customerid": 5123,
|
||
"label": "Milk Bikis",
|
||
"latitude": 11.0290, // phone fix; optional — saved address is used without it
|
||
"longitude": 77.0290,
|
||
"tenantids": [1135, 1140], // optional: what the app THINKS the customer joined
|
||
"limit": 0 // optional: max stores, 0 = all
|
||
}
|
||
```
|
||
|
||
`tenantids` is verified, never trusted: the server intersects it with the
|
||
`tenantcustomers` table. Ids the customer is not actually registered with
|
||
come back in `unregistered_tenantids` — treat that as "refresh the local
|
||
list". A list that matches nothing at all is treated as stale and all
|
||
registered stores are used.
|
||
|
||
Response:
|
||
|
||
```json
|
||
{
|
||
"label": "Milk Bikis",
|
||
"match": {
|
||
"brand": "britannia", "catalogueid": 7, "imageid": "britannia_milk_bikis_100g",
|
||
"product_name": "Milk Bikis", "size": "100 g", "variant_key": "milk_bikis",
|
||
"image": "https://…", "score": 0.94, "method": "vector+text"
|
||
},
|
||
"catalogue_variants": [ { "…same shape…": "100 g" }, { "…": "200 g" } ],
|
||
"confidence": 0.94,
|
||
"available": true,
|
||
"recommended_locationid": 20,
|
||
"stores": [
|
||
{
|
||
"tenantid": 2, "tenantname": "R Mart", "locationid": 20, "locationname": "Hopes",
|
||
"latitude": 11.01, "longitude": 77.0, "distance_km": 3.8, "open": true,
|
||
"deliveryradius": 5, "deliverymins": 30,
|
||
"recommended": true, "available": true,
|
||
"options": [
|
||
{ "productid": 200, "productname": "Milk Bikis 100g", "size": "100 g", "price": 12, "stock": 6,
|
||
"available": true, "is_variant": false, "matched_by": "imageid", "image": "…" },
|
||
{ "productid": 201, "productname": "Milk Bikis 200g", "size": "200 g", "price": 22, "stock": 3,
|
||
"available": true, "is_variant": true, "variantname": "200 g", "matched_by": "variant-of:200" }
|
||
]
|
||
},
|
||
{ "locationid": 10, "locationname": "Peelamedu", "distance_km": 0.9, "available": false, "recommended": false,
|
||
"options": [ { "productid": 100, "stock": 0, "available": false, "…": "…" } ] }
|
||
],
|
||
"unregistered_tenantids": [],
|
||
"message": "Available at 1 of your stores."
|
||
}
|
||
```
|
||
|
||
How to read it:
|
||
|
||
- `match == null` → nothing recognised; show `message` and let them retry.
|
||
`confidence` below ~0.5 → recognised but unsure; confirm the name with the
|
||
customer before showing prices. `method: "text"` means no embedding model
|
||
was involved (not configured, or it timed out) — be a little more cautious.
|
||
- `stores` is ordered **in-stock first, then nearest**. Exactly one store has
|
||
`recommended: true` — the nearest with stock — and only when `available`
|
||
is true. Stores that sell it but have nothing on the shelf are still listed
|
||
(so the customer understands why they are not recommended); stores that do
|
||
not sell it are not.
|
||
- `options` are the things that can actually go in a basket at that store —
|
||
the matched product and each of its sizes — each a real product with its
|
||
own `productid`, price and live `stock`. Use `productid` in the existing
|
||
cart/order calls exactly as you would from the catalogue screen.
|
||
- `distance_km: -1` means the distance is unknown (no fix from the phone and
|
||
no saved address, or the store has no coordinates). Do not render it as 0.
|
||
|
||
## `POST /confirm`
|
||
|
||
Sent when the customer taps a store and an option. Re-reads live stock —
|
||
nothing is cached on this path.
|
||
|
||
```json
|
||
{ "customerid": 5123, "tenantid": 1, "locationid": 10, "productid": 100, "quantity": 2,
|
||
"latitude": 11.029, "longitude": 77.029 }
|
||
```
|
||
|
||
```json
|
||
{
|
||
"ok": false,
|
||
"reason": "out_of_stock", // in_stock | insufficient_stock | out_of_stock | not_sold_here | store_not_registered
|
||
"store": { "…the store they tapped…" },
|
||
"option": { "productid": 100, "stock": 0, "…": "…" },
|
||
"requested": 2,
|
||
"alternative": { // absent when nobody has enough
|
||
"locationid": 20, "locationname": "Hopes", "distance_km": 3.8, "recommended": true, "available": true,
|
||
"options": [ { "productid": 200, "stock": 6, "price": 12, "…": "…" } ]
|
||
},
|
||
"message": "Out of stock at Peelamedu. Hopes has it (3.8 km away)."
|
||
}
|
||
```
|
||
|
||
`ok: true` → proceed to the basket. `ok: false` → show `message`; if
|
||
`alternative` is present offer it as a one-tap switch (it is the **same
|
||
product**, not another size — the customer chose a size and we do not
|
||
substitute). These are HTTP 200s: they are answers, not errors.
|
||
|
||
## `GET /stores?customerid=5123&latitude=11.029&longitude=77.029`
|
||
|
||
The customer's registered stores, nearest first, `distance_km: -1` last.
|
||
Same `ScanStore` shape as inside `stores[]` above, without options.
|
||
|
||
## Errors (HTTP status ≠ 200)
|
||
|
||
| Status | When |
|
||
|---|---|
|
||
| 400 | Missing `customerid`/`label`/ids, or a body that is not JSON. `message` says which. |
|
||
| 404 | `customerid` does not exist. |
|
||
| 503 | The catalogue database is not reachable. Retry later; the rest of the app is unaffected. |
|
||
| 500 | Anything else. Logged server-side. |
|
||
|
||
## Behind the curtain (for whoever operates it)
|
||
|
||
- **Recognition** = pgvector cosine search over every `brand_*` table in the
|
||
catalogue (each with its own index, merged), plus a word match on
|
||
`product_name`/`title`/`search_query` that settles near-ties and works on
|
||
its own when no embedding model is configured. The model is set by
|
||
`EMBEDDING_PROVIDER/MODEL/API_KEY` and **must** be the one that indexed
|
||
the catalogue — the first search checks the vector width and refuses a
|
||
mismatch by name.
|
||
- **The catalogue's model** (verified 2026-09-15 by cosine against a stored
|
||
row: 1.0000): `all-MiniLM-L6-v2`, 384-d, unit-normalised, embedding the
|
||
`search_query` column (brand + name + category + blurb + price range).
|
||
Ollama ships it as `all-minilm`; the cluster's `ollama.krow` service serves
|
||
it, so production is:
|
||
```
|
||
EMBEDDING_PROVIDER=openai
|
||
EMBEDDING_BASE_URL=http://ollama.krow.svc.cluster.local:11434/v1
|
||
EMBEDDING_MODEL=all-minilm
|
||
EMBEDDING_API_KEY=ollama # any non-empty value; Ollama ignores it
|
||
EMBEDDING_DIMENSIONS=384
|
||
```
|
||
A bare label ("Milk Bikis") scores ~0.92 against its product's stored
|
||
vector and ~0.23 against an unrelated one, which is what the 0.30 floor in
|
||
`scanService.go` is set against. If the catalogue team ever re-embeds
|
||
with another model, change `EMBEDDING_MODEL`/`DIMENSIONS` here and
|
||
nothing else.
|
||
- **Speed**: the label's vector (7 days) and the ranked catalogue hits
|
||
(30 min) are cached in Redis and in-process, so a popular product costs
|
||
one model call platform-wide. Customer, stores and catalogue are read
|
||
concurrently; the whole lookup is capped at 5 s and a slow model degrades
|
||
to a text answer instead of a spinner. Live stock is one indexed query and
|
||
is never cached.
|
||
- **Availability** is the same rule the app's catalogue screen uses:
|
||
`products.approve = 1`, `productlocations.publishedat IS NOT NULL`, stock =
|
||
live `SUM(in) − SUM(out)` of `productstocks` at that outlet, price = the
|
||
outlet's own price else the tenant's retail price.
|
||
- **No reservation.** Confirm re-reads the ledger; a hold would give the
|
||
same answer with a timer to babysit. If contention becomes real, a
|
||
Redis-backed short hold slots in at `Confirm` without changing the API.
|
||
- **Identity** is the `customerid` in the body, like every other mobile
|
||
endpoint here — there is no auth layer yet (see `SECURITY_HANDOFF.md`).
|