nearle_daily/backend_fiesta
2
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

health score

Browse Source
This commit is contained in:
abhishek
2026-09-29 22:40:36 +05:30
parent f9fb405974
commit fb859ecda1
21 changed files with 1590 additions and 703 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.env
View File

@@ -87,3 +87,8 @@ MAIL_PASSWORD=
MAIL_FROM=care@nearledaily.com
MAIL_FROM_NAME=Nearle
MAIL_CONSOLE_URL=https://app.nearledaily.com
# ── Nutrition ───────────────────────────────────────────────────────────────
# The catalogue-intelligence host behind the health score card. The customer
# app product screen reads its nutrition panel from here. Unset means no panel.
NUTRITION_BASE=https://mcp.nearle.ai.in/api

14
.env.example
View File

@@ -120,6 +120,20 @@ EMBEDDING_DIMENSIONS=0
# ── Geocoding ───────────────────────────────────────────────────────────────
# Google Geocoding when set; OpenStreetMap's Nominatim otherwise.
GEOCODER_API_KEY=
# ── Nutrition ───────────────────────────────────────────────────────────────
#
# The catalogue-intelligence service — the same host the console reads its
# health score card from. The customer app product screen gets its nutrition
# panel from here, through `getproductbyvariant`.
#
# Read server-side rather than by the app: the brand-spelling resolution below
# would otherwise have to be reimplemented in the app, and a wrong spelling
# returns a well-formed record with every figure null — indistinguishable from
# a product nobody has scored.
#
# Unset means product screens carry no nutrition panel and nothing else changes.
NUTRITION_BASE=https://mcp.nearle.ai.in/api
# ── Email ───────────────────────────────────────────────────────────────────
#
# Sending the first-password invitation a newly onboarded merchant receives.

115
docs/NUTRITION_API.md Normal file
View File

@@ -0,0 +1,115 @@
# Nutrition and health score — the app contract
`GET /live/api/v1/mob/products/getproductbyvariant?tenantid=&productid=&variantid=`
Each product in `details[]` may now carry two extra keys. Both come from the
catalogue-intelligence service (`mcp.nearle.ai.in`) — the same records behind the
health score card in the console — fetched server-side, so the app needs no
second host, no second failure mode, and no copy of the rules below.
---
## nutrition
```json
"nutrition": {
"per": "100g",
"servingsize": "1 mini (11 g)",
"items": [
{ "name": "Energy", "value": 545, "unit": "kcal" },
{ "name": "Protein", "value": 7.5, "unit": "g" }
]
}
```
- **Absent when unknown.** Not `null`, not `{}`. A missing key means "we do not
know", never "this food has no nutrition".
- `items` is never empty when `nutrition` is present.
- `per` is `"100g"` for everything the service returns today. `servingsize` is
often absent — show the basis only when it is there.
- `value` may be a decimal. `unit` is free text and may be absent.
- Rows appear only when the service stated them. A null field is omitted; a
stated zero is kept, because "no fibre" is a fact and a dash is not.
## healthscore
```json
"healthscore": {
"score": 65,
"band": "good",
"label": "Healthy",
"positives": ["Good source of protein (7.5 g per 100 g)."],
"cautions": ["High in saturated fat (14.4 g per 100 g)."],
"diettags": ["High Fiber", "Vegetarian"],
"allergens": [],
"allergensunconfirmed": true,
"caveat": "Matched to a reference product with 61% confidence — treat these figures as a guide.",
"source": { "label": "openfoodfacts", "url": "https://..." }
}
```
- **Absent when there is nothing safe to show** — unscored, not food, or no
record at all. All three read as "not rated yet".
- `band` is one of `excellent` | `good` | `fair` | `poor`, for styling. `label`
is what a shopper reads. Use the label; do not re-derive it.
- `score` is 0–100, already rounded.
- `positives`, `cautions` and `diettags` are sentences the service wrote for a
person. Render as given.
### Two rules the app MUST honour
**`caveat`, when present, has to be on screen.** It means the underlying source
match was weak — most are; the service matches down to 0.32 confidence. A
nutrition table presented as fact on a 61% match is a claim the data does not
support.
**`allergensunconfirmed: true` means an empty `allergens` list must NOT be
rendered as "contains none".** Say "not confirmed — check the packet". Silence
standing in for "none" is the one failure here that can put somebody in hospital.
A declared allergen is always sent and must always be shown.
---
## Why the judgement is server-side
The service returns a raw number and, from this endpoint, no band. Deciding which
band, whether the match is strong enough to state plainly, and whether the
product is even food is a set of rules that already exists in the console. Two
implementations would drift and disagree about the same product on two screens.
The edibility guard is the sharpest of them. The upstream per-product endpoint is
**not** gated for it: on 4 Sep 2026 it rated Godrej Hit insecticide 80/100 with
`data_status: "verified"`, and soap and shampoo both scored 37.5. Those records
now read "unavailable", and the guard stays — this tenant sells soap and
toothpaste beside its biscuits.
## Coverage today
Measured 29 Sep 2026 against tenant 1147: **6 of 15 catalogue-linked products
have nutrition**, and fewer have a score. The Patanjali ghee this work started
from has neither.
Test with **product 7101, Balaji Wafers Simply Salted** — a full panel and a
65/100 score.
```sh
go run ./scratch/nutritionproof # three real products
go run ./scratch/nutritionproof <brand> <image_id> # any product
```
## Known bad data upstream
Balaji Wafers reports `sodium_mg: 0.967` — under 1 mg per 100 g, for salted
crisps, where 500–900 mg is normal. The `Salt` figure of 0.002 g is wrong the
same way. It looks like a grams/milligrams mix-up at the source.
Fiesta passes the value through as given rather than scaling it: silently
"correcting" a food label is how wrong data becomes invisible. It will look wrong
in the app until the agent team fixes the unit.
## Configuration
`NUTRITION_BASE=https://mcp.nearle.ai.in/api`, in `.env`. Unset means neither key
is ever sent and nothing else changes. Lookups are cached six hours, capped at
three seconds, and every failure costs that product its panel rather than the
response.

136
docs/NUTRITION_DATA.md
View File

@@ -1,136 +0,0 @@
# Nutrition data — the contract between the agent team, Fiesta and the app
The customer app shows a nutrition panel on the product screen. The data comes
from the global catalogue, which the agent team fills. This is what each side
has to produce and can rely on.
---
## 1. What the app receives
`GET /live/api/v1/mob/products/getproductbyvariant?tenantid=&productid=&variantid=`
Each product in `details[]` carries:
```json
"nutrition": {
"per": "100g",
"servingsize": "30g",
"items": [
{ "name": "Energy", "value": 520, "unit": "kcal" },
{ "name": "Protein", "value": 6.5, "unit": "g" },
{ "name": "Total Sugars", "value": 22, "unit": "g" },
{ "name": "Sodium", "value": 310, "unit": "mg" }
]
}
```
Rules the app can build on:
- **The key is ABSENT when nothing is known.** Not `null`, not `{}`. A missing
key means "we do not know", never "this food has no nutrition".
- **`items` is never empty when `nutrition` is present.** A panel with no rows
is not sent, because an empty box on a product page reads as a claim.
- **`per` and `servingsize` are each optional** and often absent — see §3. Show
the basis only when it is there. Do not default it to `100g`: a wrong basis
makes every figure beneath it a misstatement rather than an unknown.
- **`value` is a number, and may be a decimal.** Saturated fat is 11.5 g as
often as 11 g.
- **`unit` is free text and may be absent.** "Servings per pack 4" has a figure
and no unit.
- **A row may have a `name` and a `value` of 0 with no unit.** That is real
label text with no number in it — "Contains permitted natural colour". Render
the name and leave the figure column blank; do not print `0`.
- Every member of a variant group carries its own panel, so switching from
500 ml to 1 L does not blank the screen.
## 2. What the agent team fills
A `nutrition` column of type **`jsonb`** on each `brand_*` table in the
catalogue database, holding exactly the object above.
```sql
ALTER TABLE brand_patanjali ADD COLUMN IF NOT EXISTS nutrition jsonb;
```
Notes:
- **The column is optional and Fiesta already handles its absence.** Catalogue
reads discover columns per table and substitute NULL for any that are missing,
so brands can be filled one at a time and a table without the column keeps
working. Nothing has to be co-ordinated with a deploy.
- **Spelling is `servingsize`**, one word, lowercase. It is the shape the app
was written against.
- **Write what the label says.** If the pack states per 100 g, `per` is `"100g"`.
If it states per serving, say so. If it states neither, omit the field rather
than assuming.
- `nutrition` and the older `nutrients` may both exist on a row. Keep both —
the console shows the lines, the app shows the panel, and they are different
readers with different needs.
## 3. Where today's data comes from, and why it is thinner
Every catalogue row currently holds nutrition as `nutrients`, a `text[]` of
display lines:
```
{"Energy 350kcal", "Protein 7g"}
```
Fiesta parses those into the same panel shape when no structured `nutrition`
exists, so the app shows figures for products the agent team has not reached
yet. That derived panel has **no `per` and no `servingsize`** — the strings
never carried them — which is the visible difference between a filled brand and
one still waiting.
Parsing rules, in `models/nutrition.go`:
| line | becomes |
|---|---|
| `Energy 350kcal` | `{Energy, 350, kcal}` |
| `Protein: 6.5 g` | `{Protein, 6.5, g}` |
| `Energy - 520 kcal` | `{Energy, 520, kcal}` |
| `Vitamin B12 1.2µg` | `{Vitamin B12, 1.2, µg}` |
| `Servings per pack 4` | `{Servings per pack, 4, ""}` |
| `Contains permitted colour` | `{Contains permitted colour, 0, ""}` — kept whole |
| `100` | dropped — a figure with no label is noise |
Structured always wins over derived where both exist.
## 4. How it reaches a shop's product
The catalogue is the source; a tenant's product is a **snapshot** taken at
import, because the catalogue is re-scraped and rows retire. Chain:
```
brand_*.nutrition (agent team)
└─ catalogue read ─ repositories/catalogueRepository.go: nutritionOf
└─ import ─ services/productService.go: catalogueFactsOf
└─ products.cataloguefacts → {"nutrition": {...}, "nutrients": [...]}
└─ endpoint ─ services/productService.go: decorateNutrition
```
`getproductbyvariant` does **not** query the catalogue database. That would be a
cross-database lookup per product on a screen a shopper is waiting on. It reads
the snapshot only.
## 5. Products imported before the snapshot existed
**This is the one thing that must be done before any of it shows.**
`cataloguefacts` is empty on every product imported before that column existed —
measured on tenant 1147: 17 products, 15 catalogue-imported, **0 with facts**.
Those products will show no nutrition no matter what the catalogue holds.
The fix is a one-off backfill, which also restores their FSSAI licence,
highlights and provider list:
```sh
go run ./scratch/cataloguefactsbackfill # dry run — prints every change
go run ./scratch/cataloguefactsbackfill apply # writes, then prints the undo
```
It touches only products with an `imageid` and a NULL `cataloguefacts`, so
re-running it is a no-op rather than a second opinion. Run it **after** the agent
team fills a brand to pick up that brand's structured panels; it is safe to run
repeatedly as more brands are filled.

10
facade/container.go
View File

@@ -49,7 +49,7 @@ type Facade struct {
// it may be nil if catalogue env vars are not configured, in which case
// catalogue endpoints will error at query time rather than at startup.
// embedder may be nil too: scan-to-order then matches on words alone.
func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder, chat utils.Chat, agentsDir, assistantWhy string, mailer utils.Mailer, mailCfg config.MailConfig) *Facade {
func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder, chat utils.Chat, agentsDir, assistantWhy string, mailer utils.Mailer, mailCfg config.MailConfig, nutritionBase string) *Facade {
// The invitation, built first because two modules need it.
//
@@ -82,8 +82,14 @@ func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder, chat
catalogueController := controllers.NewCatalogueController(catalogueService)
// Product Module
//
// The nutrition service is the catalogue-intelligence host — the same one
// behind the health score card in the console — read by the product screen
// for its nutrition panel. Nil when NUTRITION_BASE is unset, which serves
// every product screen exactly as before, without a panel.
productRepo := repositories.NewProductRepository(db)
productService := services.NewProductService(productRepo, catalogueService)
productService := services.NewProductService(
productRepo, catalogueService, services.NewNutritionService(nutritionBase))
productController := controllers.NewProductController(productService)
// Order Module

6
main.go
View File

@@ -462,8 +462,12 @@ func main() {
log.Printf("mail: sending as %s via %s", cfg.Mail.FromAddress, cfg.Mail.Address())
}
// NUTRITION_BASE is the catalogue-intelligence host — the same service the
// console reads its health score card from. Unset means product screens
// carry no nutrition panel, and nothing else changes.
f := facade.NewFacade(db.DB, db.CatalogueDB, embedder, chat,
os.Getenv("ASSISTANT_AGENTS_DIR"), cfg.Assistant.Why(), mailer, cfg.Mail)
os.Getenv("ASSISTANT_AGENTS_DIR"), cfg.Assistant.Why(), mailer, cfg.Mail,
os.Getenv("NUTRITION_BASE"))
routes.RegisterRoutes(app, f)

9
models/catalogue.go
View File

@@ -26,12 +26,9 @@ type CatalogueProduct struct {
FSSAILicense string `json:"fssai_license,omitempty"`
Highlights PGStringArray `json:"highlights,omitempty"`
Nutrients PGStringArray `json:"nutrients,omitempty"`
// The structured panel, when the catalogue carries one. Nil falls back to
// parsing Nutrients above — see NutritionFromLines.
Nutrition *NutritionPanel `json:"nutrition,omitempty"`
SearchQuery string `json:"search_query,omitempty"`
CreatedAt time.Time `json:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
SearchQuery string `json:"search_query,omitempty"`
CreatedAt time.Time `json:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
}
// CatalogueBrand describes a brand available in the catalogue DB.

164
models/healthscore.go Normal file
View File

@@ -0,0 +1,164 @@
package models
import "strings"
/*
The health score, as the customer app renders it.
The figures come from the catalogue-intelligence service — the same record the
nutrition panel comes from, and the same one behind the health score card in the
console. See services/nutritionService.go.
── Why the judgement is made here and not in the app ───────────────────────
The service returns a raw number and, from this endpoint, no band. Deciding what
that number means — which band, whether the match is sure enough to state
plainly, whether the product is even food — is a set of rules that already exists
in the console. Sending the raw number and letting the app re-derive them would
mean two implementations of the same judgement, drifting apart, disagreeing about
the same product on two screens. The rules travel with the answer instead.
*/
type HealthScore struct {
// 0–100, rounded. The service sends one decimal and nobody reads it.
Score int `json:"score"`
// "excellent" | "good" | "fair" | "poor" — for styling.
Band string `json:"band"`
// What a shopper reads, rather than what a nutritionist would call it.
Label string `json:"label"`
// Sentences the service already wrote for a person. Rendered as given.
Positives []string `json:"positives,omitempty"`
Cautions []string `json:"cautions,omitempty"`
Diettags []string `json:"diettags,omitempty"`
// Declared allergens. A false positive sends somebody to read the packet; a
// false negative sends them to hospital, so a declared one is always shown.
Allergens []string `json:"allergens,omitempty"`
// True when an EMPTY allergen list must NOT be read as "contains none".
//
// The service accepts a source match down to 0.32 confidence, and
// `data_status: "verified"` speaks to the numbers being real, not to the
// record being this product. What must never happen is silence standing in
// for "none" — which is exactly what an empty list rendered as nothing looks
// like. An app MUST say "not confirmed" rather than draw nothing here.
Allergensunconfirmed bool `json:"allergensunconfirmed,omitempty"`
// Set when the match is not sure enough to state plainly. When present the
// app must show it: a nutrition table presented as fact on a 61% match is a
// claim the data does not support.
Caveat string `json:"caveat,omitempty"`
// Where the figures came from, for a shopper who wants to check.
Source *HealthSource `json:"source,omitempty"`
}
type HealthSource struct {
Label string `json:"label"`
URL string `json:"url"`
}
// LowConfidence is the line below which a match is a guide, not a fact.
//
// The same 0.7 the console uses. Sampling 40 scored products: 37 matched below
// 0.7 and 26 below 0.5, so this fires often — which is the point.
const LowConfidence = 0.7
/*
BandFor turns a score into a band.
The service's own `health_band` wins when it sends one. It does NOT send one
from the per-product endpoint — only from the list — so for this response the
fallback is not an edge case, it is the only path, which makes these thresholds
load-bearing rather than cosmetic.
They are the SERVICE'S thresholds, not ours. Derived from its output and since
confirmed by that team in writing:
excellent >= 80
good 60 – 79.9
fair 40 – 59.9 confirmed 40, not 50
poor < 40
Delete this fallback once `health_band` is on the per-product response — it is
on their list. Until then, picking our own numbers would be one API disagreeing
with itself depending which endpoint a screen called.
*/
func BandFor(score float64, sent string) string {
switch strings.ToLower(strings.TrimSpace(sent)) {
case "excellent", "good", "fair", "poor":
return strings.ToLower(strings.TrimSpace(sent))
}
switch {
case score >= 80:
return "excellent"
case score >= 60:
return "good"
case score >= 40:
return "fair"
default:
return "poor"
}
}
// BandLabel is what a shopper reads.
func BandLabel(band string) string {
switch band {
case "excellent":
return "Very healthy"
case "good":
return "Healthy"
case "fair":
return "Okay"
default:
return "Less healthy"
}
}
/*
foodCategoryWords mean "this is food or drink".
An ALLOWLIST, and the asymmetry of the two failure modes is why. Withholding a
score on real food costs a shopper a badge they never had. Showing one on
something inedible is a different order of mistake, and the service has made it:
measured 4 Sep 2026, `GET /nutrition/Godrej/godrej_hit_spray_1101d017` returned
`health_score: 80.0, data_status: "verified"` — an "excellent" rating for
insecticide. Palmolive soap and Pantene shampoo both scored 37.5 the same way.
Re-measured 29 Sep 2026: those records now answer `unavailable`, so the purge
their team described has run. The guard stays anyway. It costs nothing when the
data is clean, and the tenant this was built for stocks soap, shampoo and
toothpaste alongside its food.
*/
var foodCategoryWords = []string{
"beverage", "drink", "juice", "water", "tea", "coffee",
"chocolate", "candy", "confection", "sweet", "dessert",
"dairy", "milk", "cheese", "butter", "ghee", "curd", "yogurt",
"snack", "biscuit", "cookie", "wafer", "chips", "namkeen",
"atta", "staple", "flour", "rice", "dal", "pulse", "grain", "cereal",
"pasta", "noodle", "bread", "bakery",
"oil", "masala", "spice", "sauce", "pickle", "jam", "honey",
"food", "nutrition", "breakfast", "fruit", "vegetable", "egg", "meat",
}
// IsEdible reports whether a score is attached to something a person eats.
//
// An unrecognised category is treated as NOT food. On screen that reads as "not
// scored yet", which is honest — we genuinely do not know — and is what most
// products show anyway.
func IsEdible(category string) bool {
value := strings.ToLower(strings.TrimSpace(category))
if value == "" {
return false
}
// "General" carries soap and household goods alongside anything else the
// scraper could not place. Ambiguous is not good enough for this decision.
if value == "general" {
return false
}
for _, word := range foodCategoryWords {
if strings.Contains(value, word) {
return true
}
}
return false
}

133
models/nutrition.go
View File

@@ -1,137 +1,52 @@
package models
import (
"regexp"
"strconv"
"strings"
)
/*
The nutrition panel, as the customer app renders it.
── Why this is a type and not a list of strings ────────────────────────────
The catalogue already carries `nutrients`, a text[] of display lines like
"Energy 350kcal". That is enough to print bullets, which is what the console
does with it today, and not enough for an app: it cannot sort by a value, show
a per-serving column beside a per-100g one, or put the unit in a different
style from the number. It also has nowhere to say what the figures are PER,
which is the one piece of context that makes the rest meaningful — 520 kcal is
a fact about a quantity, and without "per 100g" it is a fact about nothing.
So this is the shape the agent team fills and the app reads. See
docs/NUTRITION_DATA.md for the contract.
The figures come from the catalogue-intelligence service — the same one behind
the health score card in the console — and this is the shape they reach the app
in. See services/nutritionService.go for the fetch and the mapping.
── Why the field names are ugly ────────────────────────────────────────────
`servingsize`, not `serving_size` or `servingSize`. This is the shape the app
developer asked for, and an API is a promise to a client that has already been
written against it. Consistency with the rest of Fiesta — which is itself
inconsistent, `productid` beside `image_id` beside `sku_source` — is worth less
than not breaking the caller.
developer asked for, and an API is a promise to a client already written against
it. Consistency with the rest of Fiesta — itself inconsistent, `productid`
beside `image_id` beside `sku_source` — is worth less than not breaking the
caller.
*/
type NutritionPanel struct {
// What the figures are measured against: "100g", "100ml", "1 serving".
// What the figures are measured against. "100g" for everything the service
// returns today: its top-level values are per 100g, which is what the
// console's own panel prints beneath them.
Per string `json:"per,omitempty"`
// What the pack calls one serving: "30g". Separate from `Per` because a
// label routinely states both, and the app shows them in different places.
// What the pack calls one serving — "1 mini (11 g)". Absent when the
// service did not state one, rather than defaulted: a serving size is a
// claim about the food, and a guessed one is a false claim.
Servingsize string `json:"servingsize,omitempty"`
// Never nil when this panel exists — see `HasValues`. An app that receives
// `items: null` has to branch; one that receives `[]` does not, and a panel
// with no rows should not have been sent at all.
// Never nil when this panel exists — see HasValues. An app receiving
// `items: null` has to branch; one receiving `[]` does not, and a panel with
// no rows should not have been sent at all.
Items []NutritionItem `json:"items"`
}
// NutritionItem is one line of the panel.
type NutritionItem struct {
Name string `json:"name"`
// The figure. A float because saturated fat is 11.5g as often as it is 11g,
// and rounding it to please a type would be changing a label.
// The figure. A float because saturated fat is 18.7g as often as it is 19g,
// and rounding it to please a type would be editing a label.
Value float64 `json:"value"`
// "kcal", "g", "mg". Free text on purpose: the label is the authority and a
// closed list here would mean refusing to carry whatever it actually says.
// "kcal", "g", "mg". Free text on purpose: `extended_nutrients` carries its
// own units from the source, and a closed list here would mean refusing to
// carry whatever the label actually says.
Unit string `json:"unit,omitempty"`
}
// HasValues reports whether this panel is worth sending.
//
// A panel with no rows is not a panel — it is an empty box on the product page,
// which reads as "this product has no nutrition" rather than "we do not know
// yet". The endpoint omits it instead.
// A panel with no rows is not a panel — it is an empty box on a product page,
// which a shopper reads as "this food has no nutrition" rather than "we do not
// know yet". The endpoint omits it instead.
func (p *NutritionPanel) HasValues() bool {
return p != nil && len(p.Items) > 0
}
/*
nutrientLine pulls "Energy 350kcal" apart.
── Why parsing exists at all ────────────────────────────────────────────────
Every catalogue row on the platform today holds nutrition as those display
strings and nothing else. Waiting for the agent team to refill all of them
before the app can show anything would mean shipping a field that is null for
every product, for as long as that takes.
So a structured panel is used when one exists, and one is derived from the
strings when it does not. The derived panel is strictly worse — it has no `per`
and no serving size, because the strings never carried them — and it is still
the difference between an app screen with values on it and an empty one.
── What it refuses to do ───────────────────────────────────────────────────
A line it cannot read is kept WHOLE as the name, with no value and no unit,
rather than dropped or guessed at. "Contains permitted natural colour" is a
real nutrition line and it has no number in it; binning it would quietly lose
label text, and forcing a 0 into it would state something false about the food.
*/
var nutrientLine = regexp.MustCompile(`^(.*?)[\s:]*(-?\d+(?:\.\d+)?)\s*([a-zA-Zµ%]*)$`)
// NutritionFromLines derives a panel from the catalogue's display strings.
//
// Returns nil when nothing usable is found, so a caller can tell "no nutrition"
// from "a panel with no numbers in it".
func NutritionFromLines(lines []string) *NutritionPanel {
items := make([]NutritionItem, 0, len(lines))
for _, raw := range lines {
line := strings.TrimSpace(raw)
if line == "" {
continue
}
match := nutrientLine.FindStringSubmatch(line)
if match == nil {
// No number anywhere. Kept as written — see above.
items = append(items, NutritionItem{Name: line})
continue
}
name := strings.TrimSpace(strings.Trim(match[1], "-–—:"))
if name == "" {
// The whole line was a number. Nothing sensible to label it with,
// and an unnamed row on a nutrition panel is noise.
continue
}
value, err := strconv.ParseFloat(match[2], 64)
if err != nil {
items = append(items, NutritionItem{Name: line})
continue
}
items = append(items, NutritionItem{
Name: name,
Value: value,
Unit: strings.TrimSpace(match[3]),
})
}
if len(items) == 0 {
return nil
}
// No `per` and no serving size, deliberately left empty rather than guessed
// at. "100g" is the common case and it is not the only one, and a wrong
// basis is worse than an absent one — it makes every figure beneath it a
// misstatement rather than an unknown.
return &NutritionPanel{Items: items}
}

141
models/nutrition_test.go
View File

@@ -1,141 +0,0 @@
package models
import "testing"
/*
Reading the catalogue's nutrition strings.
The catalogue holds nutrition as display lines — "Energy 350kcal" — and the app
needs figures. These are the shapes those lines actually come in, and the ones
they come in when a scrape goes sideways.
The rule throughout: never invent a number, and never lose label text. A line
that cannot be read is carried whole rather than dropped, because it is
something a manufacturer printed on a packet and this code is not the authority
on what belongs on a food label.
*/
func TestAPlainLineBecomesAFigure(t *testing.T) {
panel := NutritionFromLines([]string{"Energy 350kcal"})
if !panel.HasValues() {
t.Fatal("nothing parsed")
}
got := panel.Items[0]
if got.Name != "Energy" || got.Value != 350 || got.Unit != "kcal" {
t.Fatalf("got %+v", got)
}
}
func TestTheSeparatorsThatActuallyOccur(t *testing.T) {
// Colons, multi-word names and a space before the unit are all in the wild,
// and each one used to be a whole line lost.
for _, tc := range []struct {
line string
name string
value float64
unit string
}{
{"Protein: 6.5 g", "Protein", 6.5, "g"},
{"Total Sugars 22g", "Total Sugars", 22, "g"},
{"Saturated Fat 11.5 g", "Saturated Fat", 11.5, "g"},
{"Sodium 310mg", "Sodium", 310, "mg"},
{"Energy - 520 kcal", "Energy", 520, "kcal"},
} {
panel := NutritionFromLines([]string{tc.line})
if !panel.HasValues() {
t.Errorf("%q parsed to nothing", tc.line)
continue
}
got := panel.Items[0]
if got.Name != tc.name || got.Value != tc.value || got.Unit != tc.unit {
t.Errorf("%q → %+v, want {%s %v %s}", tc.line, got, tc.name, tc.value, tc.unit)
}
}
}
func TestADecimalSurvives(t *testing.T) {
// Saturated fat is 11.5g as often as 11g. Rounding to please a type would be
// editing a food label.
panel := NutritionFromLines([]string{"Saturated Fat 11.5g"})
if panel.Items[0].Value != 11.5 {
t.Fatalf("got %v, want 11.5", panel.Items[0].Value)
}
}
func TestALineWithNoNumberIsKeptWhole(t *testing.T) {
// Real label text. Dropping it loses something a manufacturer printed;
// forcing a 0 into it states something false about the food.
panel := NutritionFromLines([]string{"Contains permitted natural colour"})
if !panel.HasValues() {
t.Fatal("the line was dropped")
}
got := panel.Items[0]
if got.Name != "Contains permitted natural colour" {
t.Fatalf("the text was mangled: %+v", got)
}
if got.Value != 0 || got.Unit != "" {
t.Fatalf("a figure was invented for a line that had none: %+v", got)
}
}
func TestAUnitlessFigureKeepsItsNumber(t *testing.T) {
// "Servings per pack 4" has a real number and no unit.
panel := NutritionFromLines([]string{"Servings per pack 4"})
got := panel.Items[0]
if got.Name != "Servings per pack" || got.Value != 4 || got.Unit != "" {
t.Fatalf("got %+v", got)
}
}
func TestPercentAndMicrogramsAreUnits(t *testing.T) {
for _, tc := range []struct{ line, unit string }{
{"Vitamin C 45%", "%"},
{"Vitamin B12 1.2µg", "µg"},
} {
panel := NutritionFromLines([]string{tc.line})
if !panel.HasValues() || panel.Items[0].Unit != tc.unit {
t.Errorf("%q → %+v, want unit %q", tc.line, panel.Items[0], tc.unit)
}
}
}
func TestBlanksAndBareNumbersAreNotRows(t *testing.T) {
// A blank is nothing. A bare "100" has no label, and an unnamed row on a
// nutrition panel is noise a shopper cannot use.
if panel := NutritionFromLines([]string{"", " ", "100"}); panel != nil {
t.Fatalf("made a panel out of nothing: %+v", panel)
}
}
func TestNoLinesMeansNoPanel(t *testing.T) {
// nil rather than an empty panel, so a caller can tell "no nutrition known"
// from "a panel that happens to be empty" — the endpoint omits the first.
if NutritionFromLines(nil) != nil {
t.Fatal("an absent panel was reported as present")
}
if NutritionFromLines([]string{}) != nil {
t.Fatal("an absent panel was reported as present")
}
}
func TestAnEmptyPanelIsNotWorthSending(t *testing.T) {
// `items: []` on a product page renders as an empty box, which reads as
// "this food has no nutrition" rather than "we do not know yet".
var absent *NutritionPanel
if absent.HasValues() {
t.Fatal("nil reported as having values")
}
if (&NutritionPanel{Per: "100g"}).HasValues() {
t.Fatal("a panel with a basis and no rows reported as having values")
}
}
func TestTheDerivedPanelDoesNotGuessItsBasis(t *testing.T) {
// The strings never carried one. "100g" is the common case and not the only
// one, and a wrong basis makes every figure beneath it a misstatement rather
// than an unknown.
panel := NutritionFromLines([]string{"Energy 350kcal"})
if panel.Per != "" || panel.Servingsize != "" {
t.Fatalf("invented a basis: per=%q servingsize=%q", panel.Per, panel.Servingsize)
}
}

11
models/product.go
View File

@@ -186,6 +186,17 @@ type Products struct {
// Set by the service, not the repository — see decorateNutrition.
Nutrition *NutritionPanel `json:"nutrition,omitempty" gorm:"-"`
// The health score, for the same product screen and from the same record.
//
// `gorm:"-"`, absent when there is nothing safe to show, and independent of
// Nutrition above — a product can be scored with no figures published, and
// carry figures with no score.
//
// WITHHELD on anything that is not food. The upstream per-product endpoint
// is not gated for edibility and has rated insecticide 80/100; see
// models.IsEdible.
Healthscore *HealthScore `json:"healthscore,omitempty" gorm:"-"`
Productdesc string `json:"productdesc,omitempty"`
Productsku string `json:"productsku,omitempty"`
Brandid int `json:"brandid,omitempty"`

41
repositories/catalogueRepository.go
View File

@@ -37,12 +37,7 @@ var ErrCatalogueDBUnavailable = errors.New("catalogue database is not configured
// catalogueProductColumns casts the text[] columns to text: GORM's raw
// scan-into-struct silently drops slice-kind destination fields, so they
// are read as text here and parsed into []string in scanProductRow.
//
// DELIBERATELY WITHOUT `nutrition`. This list is the fallback for when column
// discovery has not run, and it asserts that every column it names exists — so
// naming one that no brand table has yet would fail every read on this path
// rather than cost one field. `nutrition` lives in catalogueOptionalColumns
// below, which is what lets it ship before the agent team creates it.
const catalogueProductColumns = `id, product_name, title, description, category, image_id, size,
variant_key, product_sku, sku_source, price_range, providers::text AS providers, fssai_license,
highlights::text AS highlights, nutrients::text AS nutrients, search_query, created_at, updated_at`
@@ -73,12 +68,6 @@ var catalogueOptionalColumns = []struct{ Name, Present, Absent string }{
{"fssai_license", "fssai_license", "NULL::text AS fssai_license"},
{"highlights", "highlights::text AS highlights", "NULL::text AS highlights"},
{"nutrients", "nutrients::text AS nutrients", "NULL::text AS nutrients"},
// The structured panel the agent team fills: per, serving size and typed
// rows. Optional like everything else here, and that is what makes it
// shippable before the column exists — a brand table without it reads NULL
// and falls back to parsing `nutrients` above, rather than going dark.
{"nutrition", "nutrition::text AS nutrition", "NULL::text AS nutrition"},
{"search_query", "search_query", "NULL::text AS search_query"},
{"created_at", "created_at", "NULL::timestamptz AS created_at"},
{"updated_at", "updated_at", "NULL::timestamptz AS updated_at"},
@@ -146,7 +135,6 @@ type catalogueProductRow struct {
FSSAILicense string
Highlights string
Nutrients string
Nutrition string
SearchQuery string
CreatedAt time.Time
UpdatedAt time.Time
@@ -230,7 +218,6 @@ func (row catalogueProductRow) toModel(brand string) models.CatalogueProduct {
FSSAILicense: row.FSSAILicense,
Highlights: models.ParsePGArray(row.Highlights),
Nutrients: models.ParsePGArray(row.Nutrients),
Nutrition: nutritionOf(row),
SearchQuery: row.SearchQuery,
CreatedAt: row.CreatedAt,
UpdatedAt: row.UpdatedAt,
@@ -741,29 +728,3 @@ func (r *catalogueRepository) GetProductByID(brand string, id int64) (*models.Ca
product := row.toModel(strings.ToLower(brand))
return &product, nil
}
// nutritionOf decides which nutrition a catalogue row actually has.
//
// Two sources, and the structured one wins wherever it exists. The agent team
// fills `nutrition` — a JSON panel with the basis, the serving size and typed
// rows — and that is the one the app wants. Everything already in the catalogue
// has only `nutrients`, a text[] of display lines, so those are parsed into the
// same shape rather than left unusable.
//
// The derived panel is strictly worse: the strings never carried a basis, so it
// has no "per 100g" on it. It is still the difference between a product page
// with figures and an empty one, for every row on the platform today.
//
// A malformed `nutrition` value falls back rather than failing the read. One
// bad JSON blob in one row should cost that row its panel, not take out the
// brand listing it appears in.
func nutritionOf(row catalogueProductRow) *models.NutritionPanel {
if raw := strings.TrimSpace(row.Nutrition); raw != "" && raw != "null" {
var panel models.NutritionPanel
if err := json.Unmarshal([]byte(raw), &panel); err == nil && panel.HasValues() {
return &panel
}
log.Printf("catalogue: product %d has nutrition that could not be read; falling back to its nutrient lines", row.ID)
}
return models.NutritionFromLines(models.ParsePGArray(row.Nutrients))
}

2
routes/startup_test.go
View File

@@ -43,7 +43,7 @@ func testFacade(t *testing.T) *facade.Facade {
// No database, no catalogue, no embedder, no model. A deployment with none
// of those must still boot and say what it is missing, rather than failing
// somewhere the operator cannot see.
return facade.NewFacade(nil, nil, nil, nil, "", "no model in tests", nil, config.MailConfig{})
return facade.NewFacade(nil, nil, nil, nil, "", "no model in tests", nil, config.MailConfig{}, "")
}
func TestTheServerCanBeBuilt(t *testing.T) {

41
scratch/cataloguefactsbackfill/main.go
View File

@@ -64,17 +64,6 @@ var scalarFacts = []string{
var arrayFacts = []string{"providers", "highlights", "nutrients"}
// The structured nutrition panel: a jsonb column the agent team fills, which
// the customer app renders. Its own category because it is neither a scalar
// nor a text[] — stored as a string it would reach the app as an escaped blob
// inside the snapshot instead of an object, and every reader would have to
// decode it twice.
//
// Absent on every brand table today, which costs this fact and nothing else.
// The lines in `nutrients` above are parsed into a panel at read time, so a
// product backfilled before the column exists still shows figures.
var jsonFacts = []string{"nutrition"}
type product struct {
Productid int
Productbrand string
@@ -323,7 +312,7 @@ func factColumnsOf(db *gorm.DB, table string) []string {
}
var keep []string
for _, c := range append(append(append([]string{}, scalarFacts...), arrayFacts...), jsonFacts...) {
for _, c := range append(append([]string{}, scalarFacts...), arrayFacts...) {
if present[c] {
keep = append(keep, c)
}
@@ -343,9 +332,7 @@ func factsFor(db *gorm.DB, table string, cols []string, imageID string) (map[str
selects := make([]string, 0, len(cols))
for _, c := range cols {
// jsonb and text[] are both cast to text for the same reason: GORM's
// raw scan cannot land either one in a Go value directly.
if isArrayFact(c) || isJSONFact(c) {
if isArrayFact(c) {
selects = append(selects, c+"::text AS "+c)
continue
}
@@ -369,21 +356,6 @@ func factsFor(db *gorm.DB, table string, cols []string, imageID string) (map[str
if text == "" {
continue
}
if isJSONFact(c) {
// Embedded as an object, not as the text the column scanned to.
// Stored as a string it would reach the app as an escaped blob
// inside the snapshot, and every reader would decode it twice.
//
// Unreadable JSON is skipped rather than stored: an absent panel
// falls back to the nutrient lines at read time, a broken one does
// not.
var panel models.NutritionPanel
if err := json.Unmarshal([]byte(text), &panel); err != nil || !panel.HasValues() {
continue
}
facts[c] = panel
continue
}
if isArrayFact(c) {
values := models.ParsePGArray(text)
kept := make([]string, 0, len(values))
@@ -402,15 +374,6 @@ func factsFor(db *gorm.DB, table string, cols []string, imageID string) (map[str
return facts, true
}
func isJSONFact(name string) bool {
for _, c := range jsonFacts {
if c == name {
return true
}
}
return false
}
func isArrayFact(name string) bool {
for _, c := range arrayFacts {
if c == name {

59
scratch/nutritionproof/main.go Normal file
View File

@@ -0,0 +1,59 @@
// Proves the nutrition panel and health score end to end against the LIVE
// catalogue-intelligence service — no database, no Fiesta, just the piece that
// was built.
//
// go run ./scratch/nutritionproof # known products
// go run ./scratch/nutritionproof <brand> <image_id> # any product
//
// Prints what `getproductbyvariant` will put on the product once deployed.
package main
import (
"encoding/json"
"fmt"
"os"
"nearle/models"
"nearle/services"
)
func main() {
base := os.Getenv("NUTRITION_BASE")
if base == "" {
base = "https://mcp.nearle.ai.in/api"
}
service := services.NewNutritionService(base)
// Real products from tenant 1147, checked against the live service on
// 29 Sep 2026: one the service has scored, one it has nothing for, and one
// that is not food — which must come back with no score whatever the
// service says, because that endpoint is not gated for edibility.
products := []models.Products{
{Productid: 7101, Productbrand: "balaji", Imageid: "balaji_balaji_wafers_simply_salted_135g",
Productname: "Balaji Wafers Simply Salted 135g (scored)"},
{Productid: 7093, Productbrand: "patanjali", Imageid: "patanjali_patanjali_cow_ghee_500ml",
Productname: "Patanjali Cow Ghee 500ml (no data)"},
{Productid: 7121, Productbrand: "godrej", Imageid: "godrej_godrej_no1_sandal_and_turmeric_soap_100g",
Productname: "Godrej No.1 Soap 100g (not food)"},
}
if len(os.Args) == 3 {
products = []models.Products{{
Productbrand: os.Args[1],
Imageid: os.Args[2],
Productname: os.Args[1] + "/" + os.Args[2],
}}
}
for i := range products {
found := service.ForProduct(products[i].Productbrand, products[i].Imageid)
products[i].Nutrition = found.Panel
products[i].Healthscore = found.Health
fmt.Printf("\n---- %s ----\n", products[i].Productname)
encoded, _ := json.MarshalIndent(map[string]any{
"nutrition": products[i].Nutrition,
"healthscore": products[i].Healthscore,
}, "", " ")
fmt.Println(string(encoded))
}
}

292
services/healthScore_test.go Normal file
View File

@@ -0,0 +1,292 @@
package services
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"nearle/models"
)
/*
The health score on the product screen.
Most of this file is about what the score REFUSES to say. The number itself is
the service's; the judgements around it — which band, whether the match is sure
enough to state plainly, whether the product is even food — are decisions this
code makes on a shopper's behalf, and each one has a way of being wrong that
matters more than being absent.
The fixture is the service's own record for Balaji Wafers Simply Salted, read
from mcp.nearle.ai.in on 29 Sep 2026: a real score of 65.3 on a real product in
a real tenant's catalogue, matched at 0.607 confidence — below the line.
*/
const balajiJSON = `{
"data_status": "verified", "category": "Snacks",
"health_score": 65.3, "nutrition_score": 61.0, "match_confidence": 0.607,
"positive_insights": ["Good source of fibre (9.3 g per 100 g)."],
"nutritional_cautions": ["High in saturated fat (14.4 g per 100 g)."],
"diet_tags": ["Vegetarian"], "allergens": [],
"data_source": "openfoodfacts", "source_url": "https://world.openfoodfacts.org/product/1",
"calories_kcal": 545, "protein_g": 7.5, "total_fat_g": 30.7
}`
func healthOf(t *testing.T, raw string) *models.HealthScore {
t.Helper()
var source NutritionSource
if err := json.Unmarshal([]byte(raw), &source); err != nil {
t.Fatalf("fixture: %v", err)
}
return source.Health()
}
func TestARealScoreReachesTheApp(t *testing.T) {
score := healthOf(t, balajiJSON)
if score == nil {
t.Fatal("no score from a verified, edible, scored product")
}
if score.Score != 65 {
// Rounded: the service sends 65.3 and nobody reads the decimal.
t.Errorf("score = %d, want 65", score.Score)
}
if score.Band != "good" || score.Label != "Healthy" {
t.Errorf("band = %q / %q", score.Band, score.Label)
}
if len(score.Positives) != 1 || len(score.Cautions) != 1 {
t.Errorf("insights lost: %+v / %+v", score.Positives, score.Cautions)
}
if score.Source == nil || score.Source.Label != "openfoodfacts" {
t.Errorf("source lost: %+v", score.Source)
}
}
/* ── What it refuses to say ──────────────────────────────────────────────── */
func TestSomethingThatIsNotFoodGetsNoScore(t *testing.T) {
// The reason this guard exists. Measured 4 Sep 2026, the upstream
// per-product endpoint rated Godrej Hit insecticide 80/100 with
// data_status "verified"; soap and shampoo both scored 37.5 the same way.
// Those records now read "unavailable", and the guard stays: this tenant
// sells soap and toothpaste next to its biscuits.
for _, category := range []string{"Household", "Personal Care", "Insecticide", "Cosmetics"} {
raw := strings.Replace(balajiJSON, `"category": "Snacks"`, `"category": "`+category+`"`, 1)
if score := healthOf(t, raw); score != nil {
t.Errorf("%s was rated %d/100", category, score.Score)
}
}
}
func TestAnUnknownCategoryIsTreatedAsNotFood(t *testing.T) {
// An allowlist, because the two failure modes are not symmetric. Withholding
// a badge from real food costs a shopper something they never had; putting
// one on bleach is a different order of mistake.
for _, category := range []string{`null`, `""`, `"General"`, `"Miscellaneous"`} {
raw := strings.Replace(balajiJSON, `"category": "Snacks"`, `"category": `+category, 1)
if score := healthOf(t, raw); score != nil {
t.Errorf("category %s was scored %d/100", category, score.Score)
}
}
}
func TestAnUnscoredProductGetsNoScore(t *testing.T) {
// The common case: the service knows the product and has not rated it.
raw := strings.Replace(balajiJSON, `"health_score": 65.3`, `"health_score": null`, 1)
if score := healthOf(t, raw); score != nil {
t.Fatalf("invented a score: %+v", score)
}
}
/* ── Saying how sure it is ───────────────────────────────────────────────── */
func TestAWeakMatchIsDeclared(t *testing.T) {
// 0.607 is below the 0.7 line. A nutrition table presented as fact on a 61%
// match is a claim the data does not support.
score := healthOf(t, balajiJSON)
if score.Caveat == "" {
t.Fatal("a 0.607 match was presented as fact")
}
if !strings.Contains(score.Caveat, "61%") {
t.Errorf("the caveat does not say how sure: %q", score.Caveat)
}
}
func TestAStrongMatchNeedsNoApology(t *testing.T) {
raw := strings.Replace(balajiJSON, `"match_confidence": 0.607`, `"match_confidence": 0.94`, 1)
if score := healthOf(t, raw); score.Caveat != "" {
t.Errorf("apologised for a 94%% match: %q", score.Caveat)
}
}
func TestAnEmptyAllergenListOnAWeakMatchIsFlagged(t *testing.T) {
// The one that could actually hurt somebody. The service matches sources
// down to 0.32 confidence, and "verified" speaks to the numbers being real,
// not to the record being THIS product. Silence must not stand in for
// "contains none" — the app has to say "not confirmed" rather than draw
// nothing.
score := healthOf(t, balajiJSON)
if !score.Allergensunconfirmed {
t.Fatal("an empty allergen list on a 61% match was left to speak for itself")
}
}
func TestADeclaredAllergenIsAlwaysShown(t *testing.T) {
// A false positive sends somebody to read the packet. A false negative sends
// them to hospital. So a declared allergen survives a weak match.
raw := strings.Replace(balajiJSON, `"allergens": []`, `"allergens": ["Milk", "Soy"]`, 1)
score := healthOf(t, raw)
if len(score.Allergens) != 2 {
t.Fatalf("a declared allergen was dropped: %+v", score.Allergens)
}
if score.Allergensunconfirmed {
t.Error("a list with entries in it was flagged as unconfirmed")
}
}
func TestAnEmptyListOnAStrongMatchIsNotFlagged(t *testing.T) {
raw := strings.Replace(balajiJSON, `"match_confidence": 0.607`, `"match_confidence": 0.94`, 1)
if healthOf(t, raw).Allergensunconfirmed {
t.Error("flagged an empty list that the match actually supports")
}
}
/* ── Bands ───────────────────────────────────────────────────────────────── */
func TestTheBandThresholdsAreTheServices(t *testing.T) {
// Confirmed by that team in writing. The per-product endpoint sends no
// band, so these are the only path — picking our own would be one API
// disagreeing with itself depending which endpoint a screen called.
for _, tc := range []struct {
score float64
band string
label string
}{
{92, "excellent", "Very healthy"},
{80, "excellent", "Very healthy"},
{79.9, "good", "Healthy"},
{60, "good", "Healthy"},
{59.9, "fair", "Okay"},
{40, "fair", "Okay"},
{39.9, "poor", "Less healthy"},
{0, "poor", "Less healthy"},
} {
band := models.BandFor(tc.score, "")
if band != tc.band || models.BandLabel(band) != tc.label {
t.Errorf("%v → %q/%q, want %q/%q",
tc.score, band, models.BandLabel(band), tc.band, tc.label)
}
}
}
func TestTheServicesOwnBandWinsWhenItSendsOne(t *testing.T) {
// It does not today, from this endpoint. When it starts, its answer is the
// authority and the thresholds above become the fallback they were meant to
// be.
if got := models.BandFor(20, "excellent"); got != "excellent" {
t.Errorf("ignored the service's own band: %q", got)
}
if got := models.BandFor(20, " GOOD "); got != "good" {
t.Errorf("did not normalise the service's band: %q", got)
}
if got := models.BandFor(85, "nonsense"); got != "excellent" {
t.Errorf("trusted a band it does not recognise: %q", got)
}
}
/* ── On the wire ─────────────────────────────────────────────────────────── */
func TestTheScoreAndThePanelAreIndependent(t *testing.T) {
// A product can be scored with no figures published, and carry figures with
// no score. Tying them together would blank one because the other was
// missing.
noFigures := `{"category":"Snacks","health_score":65.3,"match_confidence":0.9}`
var source NutritionSource
if err := json.Unmarshal([]byte(noFigures), &source); err != nil {
t.Fatalf("fixture: %v", err)
}
if source.Panel() != nil {
t.Error("made a panel with no figures")
}
if source.Health() == nil {
t.Error("withheld a score because there were no figures")
}
}
func TestBothHalvesComeFromOneRequest(t *testing.T) {
// Splitting a record the service returns whole would double the traffic to
// a third party on a screen a shopper is waiting on.
var hits int
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Balaji"]}`))
return
}
hits++
_, _ = w.Write([]byte(balajiJSON))
}))
defer server.Close()
found := NewNutritionService(server.URL).ForProduct("balaji", "balaji_wafers_135g")
if hits != 1 {
t.Fatalf("made %d requests for one product", hits)
}
if !found.Panel.HasValues() || found.Health == nil {
t.Fatal("one of the two halves was lost")
}
}
func TestTheWireNamesForTheScore(t *testing.T) {
row := models.Products{Productid: 7101}
row.Healthscore = healthOf(t, balajiJSON)
encoded, err := json.Marshal(row)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var out map[string]any
if err := json.Unmarshal(encoded, &out); err != nil {
t.Fatalf("unmarshal: %v", err)
}
health, ok := out["healthscore"].(map[string]any)
if !ok {
t.Fatalf("no `healthscore` key: %s", encoded)
}
for _, key := range []string{"score", "band", "label", "caveat", "allergensunconfirmed"} {
if _, ok := health[key]; !ok {
t.Errorf("missing %q: %v", key, health)
}
}
}
func TestAProductWithNoScoreHasNoKeyAtAll(t *testing.T) {
// Not `"healthscore": null`. Most products have none.
encoded, _ := json.Marshal(models.Products{Productid: 1})
var out map[string]any
_ = json.Unmarshal(encoded, &out)
if _, present := out["healthscore"]; present {
t.Fatalf("an absent score was sent as a key: %s", encoded)
}
}
func TestSoapOnTheProductScreenCarriesNoScore(t *testing.T) {
// End to end, through the decoration the endpoint actually calls. This is
// the failure a shopper would see: a health rating on a bar of soap.
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Godrej"]}`))
return
}
_, _ = w.Write([]byte(strings.Replace(balajiJSON,
`"category": "Snacks"`, `"category": "Personal Care"`, 1)))
}))
defer server.Close()
rows := []models.Products{{Productid: 7121, Productbrand: "godrej", Imageid: "godrej_soap_100g"}}
decorateNutrition(NewNutritionService(server.URL), rows)
if rows[0].Healthscore != nil {
t.Fatalf("a bar of soap was rated %d/100", rows[0].Healthscore.Score)
}
}

148
services/nutritionResponse_test.go
View File

@@ -1,148 +0,0 @@
package services
import (
"encoding/json"
"testing"
"nearle/models"
)
/*
What the product screen receives.
The app renders a nutrition panel off `getproductbyvariant`. These are about the
contract that screen is built on — the exact field names, and what happens for
the overwhelming majority of products, which have no structured nutrition at all
and are not going to for a while.
*/
func TestTheStructuredPanelIsUsedWhereItExists(t *testing.T) {
facts := `{"nutrition":{"per":"100g","servingsize":"30g","items":[
{"name":"Energy","value":520,"unit":"kcal"},
{"name":"Protein","value":6.5,"unit":"g"}]}}`
panel := nutritionFromFacts(facts)
if !panel.HasValues() {
t.Fatal("no panel")
}
if panel.Per != "100g" || panel.Servingsize != "30g" {
t.Fatalf("basis lost: per=%q servingsize=%q", panel.Per, panel.Servingsize)
}
if len(panel.Items) != 2 || panel.Items[1].Value != 6.5 {
t.Fatalf("items wrong: %+v", panel.Items)
}
}
func TestTheOldNutrientLinesStillProduceAPanel(t *testing.T) {
// This is the case that actually matters on day one: every product on the
// platform carries display lines and nothing else. Without this the app
// ships a nutrition screen that is empty for every product in every shop.
panel := nutritionFromFacts(`{"nutrients":["Energy 350kcal","Protein 7g"]}`)
if !panel.HasValues() {
t.Fatal("the lines were not used")
}
if panel.Items[0].Name != "Energy" || panel.Items[0].Value != 350 {
t.Fatalf("got %+v", panel.Items[0])
}
}
func TestStructuredBeatsLinesWhenBothArePresent(t *testing.T) {
// The import keeps both — the console shows the lines, the app shows the
// panel. The structured one is the agent team's work and carries a basis,
// so it wins.
facts := `{"nutrients":["Energy 350kcal"],
"nutrition":{"per":"100g","items":[{"name":"Energy","value":520,"unit":"kcal"}]}}`
panel := nutritionFromFacts(facts)
if panel.Per != "100g" || panel.Items[0].Value != 520 {
t.Fatalf("the derived panel won: %+v", panel)
}
}
func TestNothingKnownMeansNoPanel(t *testing.T) {
for _, facts := range []string{"", " ", "{}", `{"highlights":["Crunchy"]}`} {
if p := nutritionFromFacts(facts); p != nil {
t.Errorf("%q produced a panel: %+v", facts, p)
}
}
}
func TestAMalformedSnapshotCostsThePanelAndNothingElse(t *testing.T) {
// One bad blob must not fail the response for the variant group it is in —
// the shopper tapped a product and is owed the screen.
if p := nutritionFromFacts(`{"nutrition":{"items":`); p != nil {
t.Fatalf("read a panel out of broken JSON: %+v", p)
}
}
func TestAPanelWithNoRowsIsNotSent(t *testing.T) {
// `items: []` renders as an empty box, which a shopper reads as "this food
// has no nutrition" rather than "we do not know".
if p := nutritionFromFacts(`{"nutrition":{"per":"100g","items":[]}}`); p != nil {
t.Fatalf("sent an empty panel: %+v", p)
}
}
func TestEveryRowInAVariantGroupIsDecorated(t *testing.T) {
// A shopper switching from 500ml to 1L must not watch the nutrition vanish.
rows := []models.Products{
{Productid: 1, Cataloguefacts: `{"nutrients":["Energy 350kcal"]}`},
{Productid: 2, Cataloguefacts: `{"nutrition":{"per":"100g","items":[{"name":"Energy","value":700,"unit":"kcal"}]}}`},
{Productid: 3},
}
decorateNutrition(rows)
if !rows[0].Nutrition.HasValues() || !rows[1].Nutrition.HasValues() {
t.Fatal("a sibling was left without its panel")
}
if rows[2].Nutrition != nil {
t.Fatal("invented a panel for a product with no facts")
}
}
func TestTheWireNamesAreTheOnesTheAppAskedFor(t *testing.T) {
// An API is a promise to a client already written against it. `servingsize`
// is not how the rest of this file would spell it, and it is what was asked
// for, which outranks house style.
rows := []models.Products{{
Productid: 7093,
Cataloguefacts: `{"nutrition":{"per":"100g","servingsize":"30g","items":[{"name":"Energy","value":520,"unit":"kcal"}]}}`,
}}
decorateNutrition(rows)
encoded, err := json.Marshal(rows[0])
if err != nil {
t.Fatalf("marshal: %v", err)
}
var out map[string]any
if err := json.Unmarshal(encoded, &out); err != nil {
t.Fatalf("unmarshal: %v", err)
}
nutrition, ok := out["nutrition"].(map[string]any)
if !ok {
t.Fatalf("no `nutrition` key on the product: %s", encoded)
}
for _, key := range []string{"per", "servingsize", "items"} {
if _, ok := nutrition[key]; !ok {
t.Errorf("missing %q: %v", key, nutrition)
}
}
item := nutrition["items"].([]any)[0].(map[string]any)
for _, key := range []string{"name", "value", "unit"} {
if _, ok := item[key]; !ok {
t.Errorf("item missing %q: %v", key, item)
}
}
}
func TestAProductWithNoNutritionHasNoKeyAtAll(t *testing.T) {
// Not `"nutrition": null`. Most products have none, and a null on every row
// of a mobile response is payload spent saying nothing.
encoded, _ := json.Marshal(models.Products{Productid: 1})
var out map[string]any
_ = json.Unmarshal(encoded, &out)
if _, present := out["nutrition"]; present {
t.Fatalf("an absent panel was sent as a key: %s", encoded)
}
}

468
services/nutritionService.go Normal file
View File

@@ -0,0 +1,468 @@
package services
import (
"encoding/json"
"fmt"
"io"
"log"
"math"
"net/http"
"net/url"
"nearle/models"
"regexp"
"sort"
"strings"
"sync"
"time"
)
/*
Nutrition for the customer app's product screen.
── Where the figures come from ─────────────────────────────────────────────
`mcp.nearle.ai.in` — the catalogue-intelligence service, the same one that
scrapes the global catalogue and backs the health score card in the console.
Not Fiesta's own database: nothing on `products` has ever held nutrition, and
the service already has it keyed by brand and image_id, which is exactly what a
tenant's product carries from import.
So this reads it rather than copying it. A second store of the same figures is a
second thing to keep in step, and the one that drifts is the one on a food label.
── Why the console does this in the browser and the app cannot ─────────────
The console calls the service directly from `api/nutrition.ts`. The customer app
could too, in principle, and should not have to: it would mean a second base
URL, a second failure mode and the brand-spelling problem below reimplemented in
whatever the app is written in. The product screen already calls Fiesta, so
Fiesta answers the whole question.
*/
// NutritionSource is the shape the catalogue-intelligence service answers with.
//
// Only the fields the panel needs. The service returns roughly forty, including
// scores, insights, allergens and diet tags — all of which belong to the health
// score card in the console and none of which the app asked for.
type NutritionSource struct {
// "verified" when the source record was confirmed, "unavailable" when the
// service knows the product and has nothing for it.
DataStatus string `json:"data_status"`
// The health score half of the same record.
HealthScore *float64 `json:"health_score"`
// Absent from this endpoint — only the list sends it — so the band is
// derived. See models.BandFor.
HealthBand string `json:"health_band"`
// What the service thinks the product IS. The edibility guard reads this,
// and it is routinely null, which the guard treats as "not food".
Category string `json:"category"`
// 0–1. How sure the service is it matched the right source record. Below
// models.LowConfidence the figures are a guide, not a fact.
MatchConfidence *float64 `json:"match_confidence"`
PositiveInsights []string `json:"positive_insights"`
NutritionalCautions []string `json:"nutritional_cautions"`
DietTags []string `json:"diet_tags"`
Allergens []string `json:"allergens"`
DataSource string `json:"data_source"`
SourceURL string `json:"source_url"`
ServingSizeLabel string `json:"serving_size_label"`
// Per 100g, every one of them. The service's own insights say so — "6.8 g
// per 100 g" — and the console prints "per 100 g" beneath the same numbers.
Calories *float64 `json:"calories_kcal"`
Protein *float64 `json:"protein_g"`
Carbohydrates *float64 `json:"carbohydrates_g"`
TotalSugar *float64 `json:"total_sugar_g"`
AddedSugar *float64 `json:"added_sugar_g"`
DietaryFiber *float64 `json:"dietary_fiber_g"`
TotalFat *float64 `json:"total_fat_g"`
SaturatedFat *float64 `json:"saturated_fat_g"`
TransFat *float64 `json:"trans_fat_g"`
Cholesterol *float64 `json:"cholesterol_mg"`
Sodium *float64 `json:"sodium_mg"`
// Whatever else the source record stated, as name → {unit, value}. The
// 5 Star record carries `{"Salt": {"unit":"g","value":0.268}}`. Carried
// through rather than filtered: it is label text, and this code is not the
// authority on what belongs on a food label.
Extended map[string]struct {
Unit string `json:"unit"`
Value float64 `json:"value"`
} `json:"extended_nutrients"`
}
// Panel maps the source record onto what the app renders.
//
// Order is the order a label prints: energy, the macros, then what the source
// added. Only fields the service actually stated — a table of dashes is worse
// than a short table, and the service genuinely omits things (`added_sugar_g`
// is null on the 5 Star record).
//
// Returns nil when nothing was stated, so "no panel" and "an empty panel" stay
// different answers.
func (s *NutritionSource) Panel() *models.NutritionPanel {
if s == nil {
return nil
}
rows := []struct {
name string
value *float64
unit string
}{
{"Energy", s.Calories, "kcal"},
{"Protein", s.Protein, "g"},
{"Carbohydrate", s.Carbohydrates, "g"},
{"Total Sugars", s.TotalSugar, "g"},
{"Added Sugars", s.AddedSugar, "g"},
{"Dietary Fibre", s.DietaryFiber, "g"},
{"Total Fat", s.TotalFat, "g"},
{"Saturated Fat", s.SaturatedFat, "g"},
{"Trans Fat", s.TransFat, "g"},
{"Cholesterol", s.Cholesterol, "mg"},
{"Sodium", s.Sodium, "mg"},
}
items := make([]models.NutritionItem, 0, len(rows)+len(s.Extended))
for _, row := range rows {
if row.value == nil {
continue
}
items = append(items, models.NutritionItem{
Name: row.name, Value: *row.value, Unit: row.unit,
})
}
// Sorted, because a Go map has no order and a nutrition panel that
// reshuffles between two requests for the same product looks broken.
for _, name := range sortedKeys(s.Extended) {
extra := s.Extended[name]
items = append(items, models.NutritionItem{
Name: name, Value: extra.Value, Unit: extra.Unit,
})
}
if len(items) == 0 {
return nil
}
return &models.NutritionPanel{
Per: "100g",
Servingsize: strings.TrimSpace(s.ServingSizeLabel),
Items: items,
}
}
func sortedKeys[V any](m map[string]V) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}
/*
NutritionService fetches one product's panel.
── Everything here is about not hurting the product screen ─────────────────
This runs inside `getproductbyvariant`, which a shopper is waiting on, and it
calls a service Fiesta does not own. So:
- a short timeout, because a slow third party must not become a slow shop;
- a cache, because nutrition for a packaged product does not change during a
trading day and a variant group asks for several products at once;
- every failure returns nil, never an error. A product page without a
nutrition panel is a product page. A product page that 500s is not.
*/
type NutritionService interface {
// ForProduct returns the nutrition panel and the health score for one
// product. Either may be nil, independently: a product can be scored with
// no figures published, and carry figures with no score.
//
// Never returns an error: see above.
ForProduct(brand, imageID string) ProductNutrition
}
// ProductNutrition is both halves of one product's record.
//
// One value because they come from ONE request. Fetching them separately would
// double the traffic to a third party on a screen a shopper is waiting on, to
// split a record the service returns whole.
type ProductNutrition struct {
Panel *models.NutritionPanel
Health *models.HealthScore
}
// nutritionTimeout is deliberately short.
//
// The alternative is a shopper watching a spinner because somebody else's
// service is having a bad afternoon. A missing panel costs a section of one
// screen; a slow response costs the screen.
const nutritionTimeout = 3 * time.Second
// nutritionTTL is how long a fetched panel is reused.
//
// A packaged product's nutrition does not change during a trading day, and the
// console's own panel uses the same reasoning. Long enough to matter, short
// enough that a correction at the source reaches shoppers the same day.
const nutritionTTL = 6 * time.Hour
type nutritionService struct {
base string
client *http.Client
mu sync.RWMutex
cache map[string]cachedPanel
brands []string
// Zero until the brand list has been read once.
brandsAt time.Time
}
type cachedPanel struct {
value ProductNutrition
at time.Time
}
// NewNutritionService builds the client, or returns nil when no base URL is set.
//
// Nil is a working configuration, like the mailer: a deployment without the
// catalogue-intelligence service still serves every product screen, without a
// nutrition panel on it. `PanelFor` is nil-safe so no caller has to check.
func NewNutritionService(base string) NutritionService {
base = strings.TrimRight(strings.TrimSpace(base), "/")
if base == "" {
return nil
}
return &nutritionService{
base: base,
client: &http.Client{Timeout: nutritionTimeout},
cache: map[string]cachedPanel{},
}
}
func (s *nutritionService) ForProduct(brand, imageID string) ProductNutrition {
if s == nil {
return ProductNutrition{}
}
brand, imageID = strings.TrimSpace(brand), strings.TrimSpace(imageID)
if brand == "" || imageID == "" {
// No join key. Sheet-imported products have no image_id, and there is
// nothing to look up rather than something that failed to be found.
return ProductNutrition{}
}
key := brand + "/" + imageID
s.mu.RLock()
hit, ok := s.cache[key]
s.mu.RUnlock()
if ok && time.Since(hit.at) < nutritionTTL {
return hit.value
}
value := s.fetch(brand, imageID)
// A miss is cached too. Most products are not scored yet, and re-asking on
// every tap would mean the least useful answer costing the most requests.
s.mu.Lock()
s.cache[key] = cachedPanel{value: value, at: time.Now()}
s.mu.Unlock()
return value
}
func (s *nutritionService) fetch(brand, imageID string) ProductNutrition {
resolved := s.resolveBrand(brand)
var source NutritionSource
if !s.get(fmt.Sprintf("/nutrition/%s/%s",
url.PathEscape(resolved), url.PathEscape(imageID)), &source) {
return ProductNutrition{}
}
return ProductNutrition{Panel: source.Panel(), Health: source.Health()}
}
/*
resolveBrand turns our spelling of a brand into theirs.
The two catalogues agree on every brand and disagree on how to write it:
ours theirs
patanjali → Patanjali
coca_cola → Coca-Cola underscore becomes a HYPHEN
brooke_bond → Brooke Bond underscore becomes a SPACE
Which separator an underscore becomes cannot be derived — hyphen for Coca-Cola
and Colgate-Palmolive, space for everything else — so the list is fetched and
matched on a normalised form rather than guessed at.
This is not cosmetic. A wrong spelling returns a well-formed record with every
figure null, which is indistinguishable from a product nobody has scored. Get it
wrong and nutrition is simply absent, everywhere, forever, with nothing in any
log to say why.
Our own spelling is returned when theirs is unknown, so a brand they have not
listed still gets a real attempt rather than being dropped.
*/
func (s *nutritionService) resolveBrand(raw string) string {
wanted := normaliseBrand(raw)
if wanted == "" {
return raw
}
s.mu.RLock()
brands, at := s.brands, s.brandsAt
s.mu.RUnlock()
// Refreshed on the same clock as a panel: a brand list changes when the
// scraper learns a new brand, which is not often and not urgently.
if at.IsZero() || time.Since(at) > nutritionTTL {
var payload struct {
Brands []string `json:"brands"`
}
if s.get("/brands", &payload) {
brands = payload.Brands
s.mu.Lock()
s.brands, s.brandsAt = brands, time.Now()
s.mu.Unlock()
}
}
for _, candidate := range brands {
if normaliseBrand(candidate) == wanted {
return candidate
}
}
return raw
}
var brandNoise = regexp.MustCompile(`[^a-z0-9]`)
func normaliseBrand(brand string) string {
return brandNoise.ReplaceAllString(strings.ToLower(strings.TrimSpace(brand)), "")
}
// get reads one JSON document. Reports whether it got one.
//
// Every failure is a false and a log line, never an error returned upward: the
// caller's job is to draw a product screen and it can do that without this.
func (s *nutritionService) get(path string, into any) bool {
response, err := s.client.Get(s.base + path)
if err != nil {
log.Printf("nutrition: %s%s: %v", s.base, path, err)
return false
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
// 404 is a normal answer here — the service does not know this product.
// Logged at the same level as the rest because a sudden wall of them is
// how a renamed path or a moved host gets noticed.
log.Printf("nutrition: %s%s: HTTP %d", s.base, path, response.StatusCode)
return false
}
// Capped: this is an upstream Fiesta does not control, and an unbounded
// read from one is how a memory limit gets found in production.
body, err := io.ReadAll(io.LimitReader(response.Body, 1<<20))
if err != nil {
log.Printf("nutrition: %s%s: %v", s.base, path, err)
return false
}
if err := json.Unmarshal(body, into); err != nil {
log.Printf("nutrition: %s%s: malformed response: %v", s.base, path, err)
return false
}
return true
}
/*
Health returns the score, or nil when there is nothing safe to show.
── The three ways this returns nothing ─────────────────────────────────────
- the service has no score for the product, which is most of them;
- the product is not food. The per-product endpoint is not gated for
edibility and has rated insecticide 80/100. See models.IsEdible — the
guard stays even though those records now read "unavailable", because the
tenant this was built for sells soap next to its biscuits;
- there is no record at all.
All three render as "not scored yet", which is honest in every case.
── What it sends that the raw record does not ──────────────────────────────
A band, a label, and — when the match is weak — a caveat and an allergen
warning. Those are judgements, and they already exist in the console. Sending
the raw number instead would mean the app re-deriving them, and two screens
disagreeing about the same product.
*/
func (s *NutritionSource) Health() *models.HealthScore {
if s == nil || s.HealthScore == nil {
return nil
}
if !models.IsEdible(s.Category) {
return nil
}
value := *s.HealthScore
band := models.BandFor(value, s.HealthBand)
// Stated plainly only when the match supports it.
var caveat string
if s.MatchConfidence != nil && *s.MatchConfidence < models.LowConfidence {
caveat = fmt.Sprintf(
"Matched to a reference product with %d%% confidence — treat these figures as a guide.",
int(math.Round(*s.MatchConfidence*100)))
}
allergens := clean(s.Allergens)
return &models.HealthScore{
Score: int(math.Round(value)),
Band: band,
Label: models.BandLabel(band),
Positives: clean(s.PositiveInsights),
Cautions: clean(s.NutritionalCautions),
Diettags: clean(s.DietTags),
Allergens: allergens,
// An empty list is only trustworthy when the match is. Silence must not
// stand in for "contains none".
Allergensunconfirmed: len(allergens) == 0 && caveat != "",
Caveat: caveat,
Source: sourceOf(s),
}
}
func sourceOf(s *NutritionSource) *models.HealthSource {
url := strings.TrimSpace(s.SourceURL)
if url == "" {
return nil
}
label := strings.TrimSpace(s.DataSource)
if label == "" {
label = "source"
}
return &models.HealthSource{Label: label, URL: url}
}
// clean drops blanks, which the service sends for an empty ai_summary and
// others.
func clean(list []string) []string {
kept := make([]string, 0, len(list))
for _, entry := range list {
if trimmed := strings.TrimSpace(entry); trimmed != "" {
kept = append(kept, trimmed)
}
}
if len(kept) == 0 {
return nil
}
return kept
}

371
services/nutritionService_test.go Normal file
View File

@@ -0,0 +1,371 @@
package services
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"nearle/models"
)
/*
The nutrition panel on the product screen.
The fixture is the catalogue-intelligence service's own record for Cadbury
5 Star, read from mcp.nearle.ai.in on 29 Sep 2026 — including the awkward parts:
a null `added_sugar_g`, a `dietary_fiber_g` of exactly 0, and an
`extended_nutrients` map carrying its own unit.
*/
const fiveStarJSON = `{
"data_status": "verified",
"serving_size_label": "1 mini (11 g)",
"serving_size_g": 11,
"calories_kcal": 525.0, "protein_g": 6.8, "carbohydrates_g": 62.6,
"total_sugar_g": 58.8, "added_sugar_g": null, "dietary_fiber_g": 0.0,
"total_fat_g": 27.9, "saturated_fat_g": 18.7, "trans_fat_g": null,
"cholesterol_mg": null, "sodium_mg": 107,
"extended_nutrients": {"Salt": {"unit": "g", "value": 0.268}}
}`
// The answer for a product the service knows nothing about — which is most of
// them, including the Patanjali ghee this work started from.
const unavailableJSON = `{
"data_status": "unavailable", "serving_size_label": null,
"calories_kcal": null, "protein_g": null, "sodium_mg": null,
"extended_nutrients": null
}`
func panelOf(t *testing.T, raw string) *models.NutritionPanel {
t.Helper()
var source NutritionSource
if err := json.Unmarshal([]byte(raw), &source); err != nil {
t.Fatalf("fixture: %v", err)
}
return source.Panel()
}
func TestARealRecordBecomesTheAppsPanel(t *testing.T) {
panel := panelOf(t, fiveStarJSON)
if !panel.HasValues() {
t.Fatal("no panel from a verified record")
}
if panel.Per != "100g" {
// The service's top-level figures are per 100g — its own insights say
// "6.8 g per 100 g" and the console prints the same basis.
t.Errorf("per = %q, want 100g", panel.Per)
}
if panel.Servingsize != "1 mini (11 g)" {
t.Errorf("servingsize = %q", panel.Servingsize)
}
want := map[string][2]any{
"Energy": {525.0, "kcal"},
"Protein": {6.8, "g"},
"Carbohydrate": {62.6, "g"},
"Total Sugars": {58.8, "g"},
"Dietary Fibre": {0.0, "g"},
"Total Fat": {27.9, "g"},
"Saturated Fat": {18.7, "g"},
"Sodium": {107.0, "mg"},
"Salt": {0.268, "g"},
}
got := map[string][2]any{}
for _, item := range panel.Items {
got[item.Name] = [2]any{item.Value, item.Unit}
}
for name, expected := range want {
if got[name] != expected {
t.Errorf("%s = %v, want %v", name, got[name], expected)
}
}
if len(panel.Items) != len(want) {
t.Errorf("got %d rows, want %d: %+v", len(panel.Items), len(want), panel.Items)
}
}
func TestAFieldTheServiceDidNotStateIsNotARow(t *testing.T) {
// `added_sugar_g` is null on this record. A table of dashes is worse than a
// shorter table, and a 0 would claim the product has no added sugar.
for _, item := range panelOf(t, fiveStarJSON).Items {
if item.Name == "Added Sugars" || item.Name == "Trans Fat" || item.Name == "Cholesterol" {
t.Errorf("invented a row the service left null: %+v", item)
}
}
}
func TestAGenuineZeroIsKept(t *testing.T) {
// `dietary_fiber_g` is 0.0, not null. "No fibre" is a fact the label states
// and dropping it would lose it — the opposite of the null case above.
var found bool
for _, item := range panelOf(t, fiveStarJSON).Items {
if item.Name == "Dietary Fibre" {
found = true
if item.Value != 0 {
t.Errorf("got %v, want 0", item.Value)
}
}
}
if !found {
t.Error("a stated zero was dropped")
}
}
func TestAnUnavailableProductGetsNoPanel(t *testing.T) {
// The common case today: the service knows the product and has nothing. An
// empty panel on the screen reads as "this food has no nutrition".
if panel := panelOf(t, unavailableJSON); panel != nil {
t.Fatalf("made a panel out of nulls: %+v", panel)
}
}
func TestTheRowOrderIsStable(t *testing.T) {
// `extended_nutrients` is a map, and Go map order is randomised. A panel
// that reshuffles between two requests for the same product looks broken.
first := panelOf(t, fiveStarJSON)
for i := 0; i < 20; i++ {
next := panelOf(t, fiveStarJSON)
for j := range first.Items {
if next.Items[j].Name != first.Items[j].Name {
t.Fatalf("row %d moved: %s then %s", j, first.Items[j].Name, next.Items[j].Name)
}
}
}
}
/* ── The client ──────────────────────────────────────────────────────────── */
func TestTheBrandSpellingIsResolvedBeforeTheLookup(t *testing.T) {
// Ours is `patanjali`, theirs is `Patanjali`. A wrong spelling returns a
// well-formed record with every figure null — indistinguishable from an
// unscored product — so this failing is silent and total.
var asked string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Patanjali","Coca-Cola","Brooke Bond"]}`))
return
}
asked = r.URL.Path
_, _ = w.Write([]byte(fiveStarJSON))
}))
defer server.Close()
service := NewNutritionService(server.URL)
if found := service.ForProduct("patanjali", "patanjali_cow_ghee_500ml"); !found.Panel.HasValues() {
t.Fatal("no panel")
}
if asked != "/nutrition/Patanjali/patanjali_cow_ghee_500ml" {
t.Fatalf("asked for %q", asked)
}
}
func TestAnUnderscoreIsNotGuessedAt(t *testing.T) {
// It becomes a hyphen for Coca-Cola and a space for Brooke Bond. Which one
// cannot be derived, so the list is matched rather than transformed.
var asked string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Coca-Cola","Brooke Bond"]}`))
return
}
asked = r.URL.Path
_, _ = w.Write([]byte(fiveStarJSON))
}))
defer server.Close()
NewNutritionService(server.URL).ForProduct("brooke_bond", "bb_tea_250g")
if asked != "/nutrition/Brooke Bond/bb_tea_250g" {
t.Fatalf("asked for %q", asked)
}
}
func TestAnUnknownBrandStillGetsATry(t *testing.T) {
// A brand they have not listed yet is attempted with our spelling rather
// than dropped — the list is their vocabulary, not a gate.
var asked string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Amul"]}`))
return
}
asked = r.URL.Path
w.WriteHeader(http.StatusNotFound)
}))
defer server.Close()
NewNutritionService(server.URL).ForProduct("newbrand", "nb_thing_1kg")
if asked != "/nutrition/newbrand/nb_thing_1kg" {
t.Fatalf("asked for %q", asked)
}
}
func TestAProductScreenSurvivesTheServiceBeingDown(t *testing.T) {
// The whole reason this returns a panel and never an error. A shopper
// tapped a product; they are owed the screen with or without nutrition.
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
}))
server.Close() // refused connections, not merely 500s
if found := NewNutritionService(server.URL).ForProduct("cadbury", "x"); found.Panel != nil {
t.Fatalf("got a panel from a dead service: %+v", found.Panel)
}
}
func TestAMalformedResponseIsNotAPanel(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":[]}`))
return
}
_, _ = w.Write([]byte(`{"calories_kcal":`))
}))
defer server.Close()
if found := NewNutritionService(server.URL).ForProduct("cadbury", "x"); found.Panel != nil {
t.Fatalf("read a panel out of broken JSON: %+v", found.Panel)
}
}
func TestTheSameProductIsNotFetchedTwice(t *testing.T) {
// This runs on a screen a shopper is waiting on, once per member of a
// variant group. Nutrition for a packaged product does not change during a
// trading day.
var hits int
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Cadbury"]}`))
return
}
hits++
_, _ = w.Write([]byte(fiveStarJSON))
}))
defer server.Close()
service := NewNutritionService(server.URL)
for i := 0; i < 5; i++ {
service.ForProduct("cadbury", "cadbury_5_star_200g")
}
if hits != 1 {
t.Fatalf("fetched %d times, want 1", hits)
}
}
func TestAMissIsCachedToo(t *testing.T) {
// Most products are unscored. Re-asking on every tap would mean the least
// useful answer costing the most requests.
var hits int
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Patanjali"]}`))
return
}
hits++
_, _ = w.Write([]byte(unavailableJSON))
}))
defer server.Close()
service := NewNutritionService(server.URL)
for i := 0; i < 4; i++ {
service.ForProduct("patanjali", "patanjali_cow_ghee_500ml")
}
if hits != 1 {
t.Fatalf("fetched %d times, want 1", hits)
}
}
func TestAProductWithNoJoinKeyIsNotLookedUp(t *testing.T) {
// Sheet-imported products have no image_id. There is nothing to look up,
// which is different from something that failed to be found.
server := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {
t.Error("called the service for a product with no image_id")
}))
defer server.Close()
service := NewNutritionService(server.URL)
service.ForProduct("cadbury", "")
service.ForProduct("", "some_image_id")
}
func TestNoServiceConfiguredIsNotACrash(t *testing.T) {
// Nil is a working configuration: a deployment without the service serves
// every product screen, without a panel.
if NewNutritionService("") != nil {
t.Fatal("built a client with no base URL")
}
rows := []models.Products{{Productid: 1, Productbrand: "cadbury", Imageid: "x"}}
decorateNutrition(nil, rows) // must not panic
if rows[0].Nutrition != nil {
t.Fatal("invented a panel with no service")
}
}
func TestEverySiblingInAVariantGroupIsDecorated(t *testing.T) {
// A shopper switching from 500ml to 1L must not watch the panel vanish.
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/brands" {
_, _ = w.Write([]byte(`{"brands":["Cadbury"]}`))
return
}
_, _ = w.Write([]byte(fiveStarJSON))
}))
defer server.Close()
rows := []models.Products{
{Productid: 1, Productbrand: "cadbury", Imageid: "a"},
{Productid: 2, Productbrand: "cadbury", Imageid: "b"},
{Productid: 3, Productbrand: "cadbury", Imageid: ""},
}
decorateNutrition(NewNutritionService(server.URL), rows)
if !rows[0].Nutrition.HasValues() || !rows[1].Nutrition.HasValues() {
t.Fatal("a sibling lost its panel")
}
if rows[2].Nutrition != nil {
t.Fatal("invented a panel for a product with no join key")
}
}
func TestTheWireNamesAreTheOnesTheAppAskedFor(t *testing.T) {
// An API is a promise to a client already written against it. `servingsize`
// is not how the rest of this codebase would spell it, and it is what was
// asked for, which outranks house style.
rows := []models.Products{{Productid: 7093}}
rows[0].Nutrition = panelOf(t, fiveStarJSON)
encoded, err := json.Marshal(rows[0])
if err != nil {
t.Fatalf("marshal: %v", err)
}
var out map[string]any
if err := json.Unmarshal(encoded, &out); err != nil {
t.Fatalf("unmarshal: %v", err)
}
nutrition, ok := out["nutrition"].(map[string]any)
if !ok {
t.Fatalf("no `nutrition` key: %s", encoded)
}
for _, key := range []string{"per", "servingsize", "items"} {
if _, ok := nutrition[key]; !ok {
t.Errorf("missing %q", key)
}
}
item := nutrition["items"].([]any)[0].(map[string]any)
for _, key := range []string{"name", "value", "unit"} {
if _, ok := item[key]; !ok {
t.Errorf("item missing %q", key)
}
}
}
func TestAProductWithNoNutritionHasNoKeyAtAll(t *testing.T) {
// Not `"nutrition": null`. Most products have none, and a null on every row
// is payload spent saying nothing.
encoded, _ := json.Marshal(models.Products{Productid: 1})
var out map[string]any
_ = json.Unmarshal(encoded, &out)
if _, present := out["nutrition"]; present {
t.Fatalf("an absent panel was sent as a key: %s", encoded)
}
}

95
services/productService.go
View File

@@ -50,10 +50,17 @@ type ProductService interface {
type productService struct {
repo repositories.ProductRepository
catalogueService CatalogueService
// May be nil. A deployment without the catalogue-intelligence service still
// serves every product screen, without a nutrition panel on it.
nutrition NutritionService
}
func NewProductService(repo repositories.ProductRepository, catalogueService CatalogueService) ProductService {
return &productService{repo: repo, catalogueService: catalogueService}
func NewProductService(
repo repositories.ProductRepository,
catalogueService CatalogueService,
nutrition NutritionService,
) ProductService {
return &productService{repo: repo, catalogueService: catalogueService, nutrition: nutrition}
}
func (s *productService) GetProductSubCategory(categoryID, tenantID int) ([]models.ProductSubCategory, error) {
@@ -251,7 +258,7 @@ func (s *productService) GetProductByVariant(tenantid, variantid, locationid, pr
// `productstock` on each row is what the app disables on. A single ungrouped
// product comes back as a one-member group, so the app has one code path:
// count the rows, and skip the picker when there is only one.
decorateNutrition(result)
decorateNutrition(s.nutrition, result)
return result, nil
}
@@ -375,19 +382,6 @@ func catalogueFactsOf(p *models.CatalogueProduct) map[string]any {
putList("highlights", p.Highlights)
putList("nutrients", p.Nutrients)
// The structured panel, snapshotted like everything else here.
//
// Kept BESIDE `nutrients` rather than instead of it. The lines are what the
// console has always shown and what a re-scrape can still change; the panel
// is what the app renders. Dropping either would break a reader that exists.
//
// `Nutrition` is already the resolved one — structured where the catalogue
// has it, derived from the lines where it does not — so a product imported
// today carries a panel whether or not the agent team has reached its brand.
if p.Nutrition.HasValues() {
facts["nutrition"] = p.Nutrition
}
return facts
}
@@ -634,61 +628,34 @@ func (s *productService) UnpublishProduct(tenantID, productID int) (int, error)
}
/*
decorateNutrition unpacks each product's nutrition panel for the app.
decorateNutrition attaches each product's nutrition panel.
── Why the endpoint does this and the query does not ───────────────────────
The panel lives inside `cataloguefacts`, a jsonb blob the import writes to hold
everything the catalogue knew that `products` has no column for. Reading it in
SQL would mean a jsonb path expression inside a query that already carries four
correlated subqueries, and it would have to be repeated in every read that ever
wants nutrition. Unpacking it once, here, keeps the query about the shelf.
The figures are not in Fiesta's database. They live in the catalogue-
intelligence service, keyed by brand and image_id — the two things a tenant's
product already carries from import — so there is nothing to join to and nothing
for the SQL to select. See nutritionService.go.
── The two shapes it accepts ───────────────────────────────────────────────
── Why a loop of network calls is acceptable here and nowhere else ─────────
`nutrition` is the structured panel and is used wherever it exists. `nutrients`
is the older text[] of display lines — "Energy 350kcal" — which is what every
product on the platform actually carries today, and it is parsed into the same
shape rather than left unreadable by the app.
This endpoint answers one tapped product plus its variant siblings: one row for
most products, a handful for a size family. Never a page of sixty. Each lookup
is cached for six hours and capped at three seconds, and a failure costs that
product its panel and nothing else.
So a product shows figures if ANYTHING is known about it, and the field is
absent only when genuinely nothing is.
── What it will not do ─────────────────────────────────────────────────────
It does not go back to the catalogue DB for a product whose snapshot is empty.
That is a cross-database lookup on a hot mobile endpoint, per product, on a
screen a shopper is waiting for. Products imported before `cataloguefacts`
existed have empty snapshots — measured: all 15 catalogue-imported products on
tenant 1147 — and the fix for those is the backfill in
`scratch/nutritionbackfill`, run once, not a lookup paid for on every tap.
It is deliberately NOT on the browse lists. Sixty products would be sixty
lookups on a screen that shows no nutrition anyway.
*/
func decorateNutrition(products []models.Products) {
func decorateNutrition(nutrition NutritionService, products []models.Products) {
if nutrition == nil {
return
}
for i := range products {
products[i].Nutrition = nutritionFromFacts(products[i].Cataloguefacts)
// One call per product, not two: the panel and the score come from the
// same upstream record.
found := nutrition.ForProduct(products[i].Productbrand, products[i].Imageid)
products[i].Nutrition = found.Panel
products[i].Healthscore = found.Health
}
}
// nutritionFromFacts reads a panel out of one product's catalogue snapshot.
//
// Every failure returns nil rather than an error: the product screen is worth
// drawing without a nutrition panel, and a malformed blob on one row must not
// fail the response for the variant group it belongs to.
func nutritionFromFacts(raw string) *models.NutritionPanel {
if strings.TrimSpace(raw) == "" {
return nil
}
var facts struct {
Nutrition *models.NutritionPanel `json:"nutrition"`
Nutrients []string `json:"nutrients"`
}
if err := json.Unmarshal([]byte(raw), &facts); err != nil {
return nil
}
if facts.Nutrition.HasValues() {
return facts.Nutrition
}
return models.NutritionFromLines(facts.Nutrients)
}

32
services/productVisibility_test.go
View File

@@ -170,7 +170,7 @@ func indexOf(calls []string, want string) int {
// showed the stock climbing and the app never listed the product.
func TestReceivingStockPutsTheProductOnTheOutletsShelf(t *testing.T) {
repo := newFakeRepo()
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
err := svc.CreateProductStock([]models.Productstock{{
Tenantid: 1135,
@@ -200,7 +200,7 @@ func TestReceivingStockPutsTheProductOnTheOutletsShelf(t *testing.T) {
// a freshly created row reading "outofstock" until the NEXT delivery.
func TestTheShelfIsCreatedBeforeAvailabilityIsDerived(t *testing.T) {
repo := newFakeRepo()
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
if err := svc.CreateProductStock([]models.Productstock{{
Tenantid: 1135, Locationid: 1170, Productid: 7086, Quantity: 25, Stocktype: "in",
@@ -225,7 +225,7 @@ func TestTheShelfIsCreatedBeforeAvailabilityIsDerived(t *testing.T) {
// correctly derived as outofstock — not be silently dropped.
func TestAnOutwardMovementAlsoShelves(t *testing.T) {
repo := newFakeRepo()
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
if err := svc.CreateProductStock([]models.Productstock{{
Tenantid: 1135, Locationid: 1170, Productid: 7086, Quantity: 3, Stocktype: "out",
@@ -245,7 +245,7 @@ func TestReimportingCorrectsAnUncategorisedProduct(t *testing.T) {
repo.existing = &models.Products{Productid: 7086, Tenantid: 1135, Categoryid: 0}
svc := NewProductService(repo, &fakeCatalogueService{
product: &models.CatalogueProduct{ID: 42, Brand: "cadbury", ProductName: "Cadbury Dairy Milk 100g"},
})
}, nil)
err := svc.ImportCatalogueProduct([]models.ImportCatalogueProductRequest{{
Tenantid: 1135, Locationid: 1170, Brand: "cadbury", Catalogueid: 42,
@@ -272,7 +272,7 @@ func TestReimportWithoutACategoryLeavesTheExistingOneAlone(t *testing.T) {
repo.existing = &models.Products{Productid: 7043, Tenantid: 1135, Categoryid: 2}
svc := NewProductService(repo, &fakeCatalogueService{
product: &models.CatalogueProduct{ID: 9, Brand: "generic", ProductName: "Apple"},
})
}, nil)
err := svc.ImportCatalogueProduct([]models.ImportCatalogueProductRequest{{
Tenantid: 1135, Locationid: 1170, Brand: "generic", Catalogueid: 9,
@@ -383,7 +383,7 @@ func (f *fakeProductRepo) GetProducts(params models.ProductFilter) ([]models.Pro
// hits and asserts on what a shopper would be shown.
func TestTheBrowseEndpointOffersOnlyStockedProducts(t *testing.T) {
repo := newFakeRepo()
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
result, err := svc.GetProductsBySubcategory(models.ProductFilter{
CategoryID: 2, TenantID: 1135, LocationID: 1170,
@@ -434,7 +434,7 @@ func TestOrderingAnUngroupedProductAsksByProductId(t *testing.T) {
repo := &variantRepo{rows: []models.Products{
{Productid: 7086, Productname: "Cadbury Dairy Milk 100g", Productstock: 25},
}}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
got, err := svc.GetProductByVariant(1135, 0, 1170, 7086)
if err != nil {
@@ -455,7 +455,7 @@ func TestOrderingAGroupedProductOffersEveryVariant(t *testing.T) {
{Productid: 6995, Productname: "Strawberries 250g", Productstock: 4},
{Productid: 6996, Productname: "Strawberries 500g", Productstock: 9},
}}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
got, err := svc.GetProductByVariant(1087, 44, 1097, 0)
if err != nil {
@@ -478,7 +478,7 @@ func TestTheProductScreenStillShowsAnEmptyProduct(t *testing.T) {
repo := &variantRepo{rows: []models.Products{
{Productid: 7085, Productname: "Cadbury 5 Star 18g", Productstock: 0},
}}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
got, err := svc.GetProductByVariant(1135, 0, 1170, 7085)
if err != nil {
@@ -502,7 +502,7 @@ func TestTheWholeFamilyIsShownIncludingEmptyOnes(t *testing.T) {
{Productid: 6994, Productname: "Pineapple", Productstock: 50},
{Productid: 6989, Productname: "Jammu Apple", Productstock: 0},
}}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
got, err := svc.GetProductByVariant(1087, 36, 1097, 7014)
if err != nil {
@@ -526,7 +526,7 @@ func TestTheWholeFamilyIsShownIncludingEmptyOnes(t *testing.T) {
// still does not. The two endpoints deliberately differ.
func TestBrowseStillHidesWhatTheShopDoesNotHave(t *testing.T) {
repo := newFakeRepo()
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
result, err := svc.GetProductsBySubcategory(models.ProductFilter{
CategoryID: 2, TenantID: 1135, LocationID: 1170,
@@ -565,7 +565,7 @@ func TestImportPublishesWhatItPriced(t *testing.T) {
ID: 1, Brand: "kohinoor", ProductName: "Kohinoor Charminar Rice 5kg",
ImageID: "kohinoor_kohinoor_charminar_rice_5kg",
},
})
}, nil)
if err := service.ImportCatalogueProduct([]models.ImportCatalogueProductRequest{{
Tenantid: 1147, Locationid: 1185, Brand: "kohinoor", Catalogueid: 1,
@@ -592,7 +592,7 @@ func TestImportPublishesAfterTheRowsExist(t *testing.T) {
repo := newFakeRepo()
service := NewProductService(repo, &fakeCatalogueService{
product: &models.CatalogueProduct{ID: 1, Brand: "colin", ProductName: "Colin Glass Cleaner 500ml"},
})
}, nil)
if err := service.ImportCatalogueProduct([]models.ImportCatalogueProductRequest{{
Tenantid: 1147, Locationid: 1185, Brand: "colin", Catalogueid: 1,
@@ -789,7 +789,7 @@ func TestPricingFilterDoesNotDisturbTheCallersSlice(t *testing.T) {
func TestCreateProductReturnsTheIdTheDatabaseAssigned(t *testing.T) {
repo := &fakeProductRepo{}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
created, err := svc.CreateProduct(models.Products{
Tenantid: 9001,
@@ -818,7 +818,7 @@ func TestCreateProductGoesThroughTheOneCreatePath(t *testing.T) {
// discarded the id. Only one remains, and this is what pins that: a second
// path would have to be added here to be used at all.
repo := &fakeProductRepo{}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
if _, err := svc.CreateProduct(models.Products{Tenantid: 9001}); err != nil {
t.Fatalf("CreateProduct: %v", err)
@@ -834,7 +834,7 @@ func TestAFailedCreateReturnsNoProduct(t *testing.T) {
// product with a zero id would be worse than an error — it would send a
// price and a stock movement to product 0.
repo := &fakeProductRepo{createErr: errors.New("duplicate key")}
svc := NewProductService(repo, &fakeCatalogueService{})
svc := NewProductService(repo, &fakeCatalogueService{}, nil)
created, err := svc.CreateProduct(models.Products{Tenantid: 9001, Productsku: "DUP"})
if err == nil {