nearle_daily/catalogue_backend
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Backend fix checkbox

Browse Source
This commit is contained in:
sriram
2026-09-01 11:25:04 +05:30
parent a6e78fee95
commit 183b65b3bd
4 changed files with 39 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
app/api/routers/batch_catalog.py
View File

@@ -10,20 +10,25 @@
POST /api/admin/catalog-batch/from-inbox - run selected files
POST /api/admin/catalog-batch/inbox/dismiss - discard selected files
This is the multi-file sibling of `store_catalog.py`, and it deliberately does
not replace it: the single-file endpoints are untouched and still work. What is
different is the unit of work. Five files are one batch with one id, so the
question a colleague actually asks - "did the drop land?" - has one answer
rather than five.
These began as the multi-file sibling of a single-file admin uploader
(`store_catalog.py` and its Store Catalog Ingestion tab). THAT UPLOADER AND ITS
ROUTES HAVE SINCE BEEN DELETED - do not go looking for them - and this router is
now the only admin way in. What made it the survivor is the unit of work: five
files are one batch with one id, so the question a colleague actually asks -
"did the drop land?" - has one answer rather than five.
WHY THE NETWORK STAGES DEFAULT OFF HERE
---------------------------------------
The single-file UI sends `use_llm=true, fetch_images=true`. At one file that is
a considered trade. At twenty it is thousands of outbound requests and, for the
image stage, a Playwright subprocess that can burn three minutes on its own -
on a single-vCPU container that is also serving the API. So a batch opts IN to
those stages; it does not opt out. `USE_OLLAMA` is false in production anyway,
which makes `use_llm` a no-op there and the honest default obvious.
Turning both on for a single file is a considered trade. At twenty files it is
thousands of outbound requests and, for the image stage, a Playwright subprocess
that can burn three minutes on its own - on a single-vCPU container that is also
serving the API. So a batch opts IN to those stages; it does not opt out.
`USE_OLLAMA` is false in production anyway, which makes `use_llm` a no-op there
and the honest default obvious.
Note that these defaults bind THIS router only. The open upload endpoint runs
itself and takes its two flags from `UPLOAD_AUTORUN_FETCH_IMAGES` and
`UPLOAD_AUTORUN_USE_LLM`, both true - see the section below.
THE OTHER WAY INTO THE SAME PIPELINE
------------------------------------
@@ -54,9 +59,10 @@ from app.infrastructure.settings import (
router = APIRouter(prefix="/admin/catalog-batch", tags=["admin", "catalog"])
# Per-file ceilings match store_catalog.py exactly. A file that is too big for
# the single-file endpoint is not somehow acceptable because it arrived with
# four friends.
# Per-file ceilings. They bound ONE spreadsheet, and they hold whether it
# arrived alone or with nineteen friends: a file too big to parse safely does
# not become acceptable by being part of a batch. The batch-wide ceilings
# (BATCH_MAX_*, imported above) then bound the set on top of these.
MAX_UPLOAD_BYTES = 10 * 1024 * 1024
MAX_UPLOAD_ROWS = 2000
PREVIEW_ROWS = 10