catalogue_backend/README.md

Backend - Brand Product Search Engine (RAG API)

FastAPI service exposing the product catalog through:

• Browse - plain listing endpoints (/api/brands, /api/brands/{brand}/products)
• Search - pgvector semantic similarity search, no LLM (/api/search)
• Chat - full RAG: retrieval + local Ollama generation (/api/chat)
• Admin - trigger brand ingestion in the background (/api/catalog/generate)

Full setup, architecture, and troubleshooting steps are in the project documentation (docs/). This file is just a fast local reference.

Quick start

One command (from the project root)

python run_project.py --backend-only

Picks up backend/venv if present (else the current interpreter) and starts uvicorn with autoreload on port 8000. Drop --backend-only to run the React frontend alongside it; --help lists the port and reload flags.

It does not start Postgres or Ollama for you - it reports them via /api/health and warns if either is unreachable. Bring those up first (see Manual below, and "Pulling the local LLM").

Manual

cd backend
python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env               # then edit DB_PASSWORD etc.

# Auth is required: the app will not start without AUTH_SECRET_KEY and the two
# password hashes. This prints them, plus the sign-in passwords (shown once).
python scripts/make_auth_secrets.py

# Option A: already have a Postgres+pgvector catalog from the old project?
#   Just point .env at it (DB_HOST/DB_PORT/DB_NAME/DB_USER/DB_PASSWORD) - done.
# Option B: starting fresh locally?
docker compose -f docker-compose.yml up -d
python scripts/seed_sample_data.py     # loads bundled sample catalogs instantly

uvicorn app.main:app --reload --port 8000

Then open http://localhost:8000/docs for interactive API docs, or run the frontend (../frontend/README.md) to use the React UI.

Authentication

Reads are public; the 18 write/compute endpoints require a credential, enforced by a dependency on each route (app/api/deps.py). Sign in for a bearer token:

curl -X POST localhost:8000/api/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"username":"admin","password":"<from make_auth_secrets.py>"}'

Send it as Authorization: Bearer <token>, or use an X-API-Key from the API_KEYS setting for server-to-server callers. admin passes every permission check; user holds the product/store/inventory permissions. AUTH_ENABLED=false disables all of it for local work — never in a deployment. See the Authentication section of ../DEPLOYMENT.md for the full endpoint map.

Persistence: the two volumes a deployment needs

Most state lives in Postgres, but three things are written to the filesystem, and in a container those live inside the image - so a redeploy rebuilds the image and silently discards them:

Path	Written by
/app/data/seed_catalogs	POST /api/user/products/add, /upload-file - every product added through the UI is appended to the brand's JSON
/app/app/intelligence/artifacts	the training endpoints - every retrained *.joblib model
/app/data	catalogs saved by the ingestion pipeline

Mount a volume on each (the first is inside the third, so two mounts cover all three):

/app/data
/app/app/intelligence/artifacts

In Dokploy, add both under the service's Volumes. Named volume or bind mount, either is fine - docker compose --profile full up -d shows the same two mounts as named volumes.

Bind mounts normally break this pattern, because they start empty and hide the seed catalogs and pre-trained models the image ships with. They are safe here: the image keeps read-only copies at /app/.bundled, and on startup app/infrastructure/persistence.py copies in whatever the mounted directory is missing. It never overwrites an existing file, so a user-added product always survives the next redeploy rather than being reverted to the bundled catalog.

If you leave the volumes off, the API still runs and logs a warning naming the directories that will be lost.

Override the locations with DATA_DIR, SEED_CATALOG_DIR and MODEL_ARTIFACTS_DIR if the writable data belongs somewhere else.

Pulling the local LLM (one-time)

ollama pull qwen2.5:1.5b
ollama serve   # if not already running as a service

Project layout

backend/
├── app/
│   ├── main.py                  # FastAPI app + router wiring
│   ├── infrastructure/settings.py
│   ├── api/
│   │   ├── schemas.py           # Pydantic models
│   │   ├── job_store.py         # in-memory background-job tracker
│   │   └── routers/             # health, brands, search, chat, catalog
│   ├── core/
│   │   ├── catalog_engine.py    # discovery + enrichment + image pipeline
│   │   └── ingestion.py         # thin wrapper used by API + CLI
│   └── services/
│       ├── embeddings_service.py  # sentence-transformers (lazy-loaded)
│       ├── ollama_service.py      # local LLM calls (catalog + RAG answers)
│       ├── vector_store.py        # pgvector reads/writes/SEMANTIC SEARCH
│       ├── rag_service.py         # RAG orchestration (NEW)
│       ├── image_search.py / s3_service.py / price_estimator.py / brand_registry.py
├── cli/ingest_brand.py           # CLI: ingest one brand end-to-end
├── scripts/seed_sample_data.py   # load bundled sample catalogs (no LLM needed)
├── data/seed_catalogs/*.json     # bundled sample catalogs (Parle, Cadbury, ...)
└── requirements.txt

Running tests

pytest -q