upload-catalog-integration

docs/INGESTION_API.md

@@ -158,10 +158,26 @@ run; it is waiting behind a decision, not behind the worker.
 `batch_id` is a `uuid4` handed only to whoever sent the drop, so holding it is the
 proof of having sent it. An id nobody issued is a `404`.
 
-Expect `pending` until an admin acts. After that the batch you sent stops existing
-under that id — the files move into a run with an id of its own, and your drop
-reports `404`. **A `404` after `pending` means your files were accepted and started,**
-not that they were lost. Ask the admin for the run id if you need to follow it.
+**Your drop id stays valid for the whole lifecycle.** Poll it and read the per-file
+`status`:
+
+| File `status` | `released_to` | Meaning |
+| --- | --- | --- |
+| `queued` | `null` | Still in the inbox; nobody has looked yet |
+| `released` | the run id | Accepted and started. **Follow that id for progress and results** |
+| `dismissed` | `null` | An admin declined this file |
+
+```jsonc
+// GET /api/uploads/catalog/{drop_id}, after an admin pressed Start
+{ "status": "retired",
+  "files": [ { "filename": "catalog.csv",
+               "status": "released",
+               "released_to": "8dcef8a2ad94..." } ] }
+```
+
+Then `GET /api/uploads/catalog/{released_to}` — also with no credential — for the run
+itself. A run an admin assembled from several drops lists every file in it, so you may
+see filenames batched alongside your own.
 
 `GET /api/uploads/catalog?limit=20` — your recent submissions, newest first, under
 `{"batches": [...]}`. This one **does** need a credential: letting an anonymous caller
@@ -173,6 +189,7 @@ hold.
 | Status | Meaning |
 | --- | --- |
 | `pending` | In the review inbox. Nothing has run. |
+| `retired` | Every file in this drop has been released or dismissed. Read the per-file `status` |
 | `queued` | Approved and waiting for the worker |
 | `running` | In the pipeline now |
 | `done` | Every file completed |
@@ -181,6 +198,68 @@ hold.
 | `interrupted` | A restart cut a run short. An admin resumes it; it never auto-restarts |
 | `cancelled` | Stopped by an admin before the remaining files began |
 
+### What a finished run tells you
+
+Each file in a completed run carries a `result`. Alongside the counts it lists **every
+row that resolved**, with the identifiers needed to reconcile the sheet against the
+catalogue:
+
+```jsonc
+"result": {
+  "rows_total": 2, "inserted": 1, "backfilled": 0, "skipped_existing": 1,
+  "rejected": 0, "brands": ["amul"],
+  "products": [
+    { "image_id": "amul_amul_butter_100g", "brand": "amul",
+      "product_name": "Amul Butter 100g",
+      "product_sku": "ACME-BUT-100", "sku_source": "sheet",
+      "disposition": "inserted" },
+    { "image_id": "amul_amul_ghee_1l", "brand": "amul",
+      "product_name": "Amul Ghee 1L",
+      "product_sku": "AMUL-GHE-1-001", "sku_source": "Internal",
+      "disposition": "unchanged" }
+  ],
+  "products_truncated": false
+}
+```
+
+| `disposition` | What happened |
+| --- | --- |
+| `inserted` | New product, created by this run |
+| `backfilled` | Existed already; this sheet filled columns it had left empty |
+| `unchanged` | Existed already and was complete. Nothing was written |
+
+**Join on `image_id`.** It is the key the catalogue deduplicates every product by. Do
+not match on `product_name` — a name differing by one character is a different
+`image_id` and therefore a different product, so name matching silently creates
+duplicates instead of updating.
+
+`unchanged` rows are listed too. Re-sending a sheet is the normal case and writes
+nothing; without them you would get an empty list back for a completely successful
+upload. `products_truncated` is `true` when a file resolved more than 5,000 rows
+(`MAX_REPORTED_PRODUCTS`) — the counts stay accurate, only the listing is cut.
+
+`products` is returned on the **single-batch read** only. The list endpoints omit it,
+since twenty runs of several thousand rows each is not a list payload.
+
+### Which SKU wins
+
+| Your sheet's `sku` column | Result | `sku_source` |
+| --- | --- | --- |
+| Has a value | **Preserved verbatim.** The resolver is never called | `sheet` |
+| Blank | A deterministic internal SKU is minted, e.g. `AMUL-GHE-1-001` | `Internal` |
+| Blank, and marketplace lookup enabled | A real marketplace product id | the marketplace name |
+
+The third row does not occur in production: `ENABLE_SKU_WEB_LOOKUP` is `false` there,
+so a blank cell always yields an `Internal` SKU.
+
+If your sheet carries its own `sku_source` column, that is preserved too and not
+overwritten with `sheet`.
+
+**A product's SKU is stable once stored.** Re-sending the same product does not
+renumber it — a backfill keeps the stored value rather than the one that run minted.
+But that stability is per `image_id`: change the product name and you get a new
+`image_id`, a new product, and a new SKU. Another reason to join on `image_id`.
+
 ### The eleven stages
 
 1. Brand Resolution & FSSAI Licence Mapping
@@ -344,6 +423,8 @@ Sample sheets with the correct headers: `GET /api/upload/template/stores`, `/ana
 | --- | --- |
 | `POST /api/uploads/catalog` needed an `X-API-Key` | No credential. Send the file; drop the header |
 | Files ran on arrival | Files wait for review. Expect `pending`, not `queued` |
+| The drop id 404'd once an admin started it | It stays valid. The file reads `released` and carries `released_to` |
+| The result was counts only | It also lists `products` with `image_id` / `product_sku` / `disposition` |
 | `?use_llm` / `?fetch_images` on the drop | Ignored. The admin chooses at Start |
 | `429` meant the worker queue was full | `429` now means the review inbox is full |
 | `200` with `rows_imported: 0` | `422` with per-row reasons. Handle as a client error, not a server one |