app/services/enrichment/barcode/retry.py imports tenacity at module scope,
and that module is on app/main.py's import path (main -> store_catalog router
-> store_catalog_pipeline -> barcode enrichment -> sources -> retry). tenacity
was in no requirements file, so the deployed image exited 1 during startup:
File "/app/app/main.py", line 33, in <module>
from app.api.routers import store_catalog
...
File "/app/app/services/enrichment/barcode/retry.py", line 16
from tenacity import (
ModuleNotFoundError: No module named 'tenacity'
This surfaced as "100% CPU", not as a crash, which is why it was mis-read.
uvicorn never bound a socket, Swarm restarted the task, and each restart
re-ran the ~21s of eager pandas/scipy/sklearn imports that the analytics,
recommendations and nutrition routers pull in at module scope. On this
1-vCPU host that loop pins the only core indefinitely.
retry.py's own docstring asserted tenacity was "already a project dependency
(see requirements.txt)". It never was - corrected to say the opposite, and to
record that it is a hard startup dependency rather than an optional extra.
Verified on the deployed image, not just locally: with tenacity present the
container reaches health=healthy with restarts=0, and /, /api/health,
/api/brands, /api/system/status and /docs all return 200. Idle cost is
0.16% CPU / 168MiB. An AST scan of every import in app/, cli/, scripts/ and
serve.py against the image finds no other missing module (playwright is the
one remaining absence and is deliberate - commented out in requirements.txt
and imported lazily inside a function).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7bVBpxH4AbJtK7Kp3MzDR
The image was 2.45GB, of which the venv is 1.73GB. The build was already
multi-stage and already installed CPU-only torch, so the remaining weight was
not build tooling - it was payload inside the installed packages that the
running service can never execute.
Removed in the build stage, before the runtime stage copies /opt/venv, so the
bytes never enter the final image:
- bundled test suites (~237MB; torch/test is 83MB, pandas/tests 40MB)
- torch/include (62MB), C++ headers for compiling against libtorch
- torch/bin (50MB), gtest binaries and protoc; torch_shm_manager is kept
pytest and httpx2 move to requirements-dev.txt: the image copies app/, cli/,
scripts/, data/ and serve.py, never tests/, so the test stack was unusable
there regardless.
sympy was checked and deliberately kept - 'import sentence_transformers' does
pull it in through torch.fx, so removing it would break embeddings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
_ensure_client() returns None - not False - when USE_OLLAMA is false, because
it returns before it ever probes. SystemStatusOut.ollama_connected is typed
bool, so pydantic rejected the None and the endpoint answered 500 on exactly
the configuration this deployment runs.
Found by smoke-testing the live host: every other read route answered 200 and
this one alone was a server error, which read like a database problem and was
not one. "Ollama is off" now reports as ollama_connected: false.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The frontend loaded but could not call the API at all: every preflight from the
browser came back with no Access-Control-Allow-Origin header, so each request
was blocked client-side while the server logged nothing wrong.
API_CORS_ORIGINS was set to catalogue.nearle.ai.in. The host Traefik actually
serves is catalouge.nearle.ai.in - the o and u transposed. The correctly spelled
domain does not resolve at all, which is why checking the "frontend" only ever
returned a connection error and looked like a network problem.
Both spellings are now listed, so this keeps working if the typo is corrected in
Dokploy later. Verified against the live API: a preflight from
https://catalouge.nearle.ai.in previously returned 400 with no allow-origin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured inside the image: 2.7GB of nvidia/ CUDA libraries and 691MB of
triton/, on a single-CPU VPS with no GPU. sentence-transformers pulls torch in
transitively, and pip's default Linux wheel bundles the whole CUDA stack because
it cannot know the target has none. That is ~3.4GB of code that can never
execute, in a 9.2GB image on a 48GB disk shared with a dozen other services -
and a build here has already failed once on "no space left on device".
torch now installs first from PyTorch's CPU index, so the requirements.txt pass
finds it satisfied and leaves it alone.
playwright is commented out rather than deleted. It is the last-resort
image-search tier and the Dockerfile never installs its browser binary, so in a
container the tier is skipped at runtime regardless while the package still
costs 137MB. Its import is lazy, so absence takes the existing "not installed"
path rather than breaking anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The container was never getting its configuration, so it started with nothing
set and Traefik reported a Bad Gateway on every route.
Dokploy writes its own .env into the build context from the service's
Environment tab AFTER cloning the repository. That tab is empty, so it wrote a
zero-byte file over the committed one, and `COPY .env .` faithfully copied the
empty result into the image. The checkout showed it exactly: every file
timestamped 08:33, and .env alone at 08:34 with a size of 0. Inside the running
container, /app/.env was 0 bytes.
Nothing about this is visible from the outside. The build log shows the COPY
succeeding, the image is produced, and the platform reports only a 502.
Dokploy does not manage .env.production, so the config now travels under that
name and the Dockerfile copies it to /app/.env in the image. Anything set in the
Environment tab still wins at runtime, because settings.py calls load_dotenv()
without override=True.
Verified by reconstructing the build context the way Dokploy does - git archive
of HEAD, then an empty .env written over it - applying .dockerignore and the
COPY lines, and booting the result with an empty environment: /app/.env is 4938
bytes, the sign-in passwords are absent, and scripts/check_deploy.sh reports 6
passed, 0 failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Separates three failures that are indistinguishable from a browser, and which
this deployment hit in sequence:
502 on every route the container is not running - it exited at startup,
so nothing reached the app and no route is special
200 + database:false the API is healthy, Postgres is not
200 + database:true working
It also catches AUTH_ALLOW_ANY_LOGIN being left on, by asserting that a
deliberately wrong password is rejected. That setting is a convenience locally
and a total auth bypass on a published host, and nothing else about a running
deployment looks different when it is true.
./scripts/check_deploy.sh
./scripts/check_deploy.sh http://localhost:3000
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The configuration was committed but excluded twice over: .dockerignore listed
.env, and the Dockerfile's explicit COPY lines never mentioned it. So the image
built cleanly, the container started with no configuration at all, and exited on
the first required setting - the same RuntimeError and the same Bad Gateway that
committing .env was meant to fix.
Silent in both directions. The build log shows every COPY succeeding, and the
platform reports only a 502, because the process is gone before it can say
anything. Nothing about "build completed" hints that the container has no
configuration.
Verified by reconstructing the image filesystem from the Dockerfile's COPY
lines with .dockerignore applied, then booting from it with an empty
environment: /app/.env is present, SIGNIN_PASSWORDS.txt is not, and /, /docs and
/api/health all answer 200. That reconstruction is the check the earlier "boots
from .env alone" testing was missing - it ran on the host, where .env sits next
to app/ whether the image would have contained it or not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two faults compounded into one symptom: with the database unreachable, every
route on the service returned 502 - including /docs, which never touches it.
_connect() passed no connect_timeout. A host that DROPS packets rather than
refusing them, which is what a firewall or a wrong DB_HOST looks like, blocked
until the OS gave up - roughly 130 seconds on Linux. Every caller inherited
that, /api/health included. Now bounded by DB_CONNECT_TIMEOUT_SECONDS,
defaulting to 5. Measured against an unroutable host: /api/health went from
hanging past 25s to answering 200 in 5.07s.
The container healthcheck then probed /api/health, so that hang timed out the
check, the container was marked unhealthy, and the platform stopped routing to
it. That is the part that turned a degraded dependency into a total outage, and
it was introduced with the healthcheck itself.
A healthcheck is a LIVENESS question, because the platform's answer to "no" is
to take the container out of service. It may only ask whether the process is
still serving HTTP. /api/health is a READINESS report - it dials Postgres and
Ollama to say whether they are reachable, and coupling the container's
existence to its dependencies is what made a running API unreachable. It now
probes "/", which is served from memory and does no I/O, so it can fail only if
the app really is gone.
Verified with an unroutable DB host: /, /docs and /openapi.json all answer 200,
and the healthcheck exits 0. With the app stopped it still exits 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the live database, DigitalOcean Spaces and Google CSE settings supplied by
the repo owner, so the container needs nothing set in the Dokploy UI.
Three deliberate departures from the development .env this came from:
AUTH_ALLOW_ANY_LOGIN is false, not true. In development it is a convenience -
the password field is not checked, so any username signs in and `admin` reaches
the admin pages. On a host published to the internet it means anyone who finds
mcp.nearle.ai.in becomes admin by typing anything at all. The development file's
own comment says to turn it off before the backend leaves the laptop.
The auth secrets are the freshly generated ones, not the development values.
Those hashes are for the passwords DevAdmin!2026 and DevUser!2026, which sit in
plaintext in test_login_fix.py in this same repository - committing them would
have published working admin credentials alongside the hash that accepts them.
Verified: DevAdmin!2026 is now rejected with a 401.
USE_OLLAMA is false. The development value http://localhost:11434 cannot work
from inside a container, where localhost is the container rather than the VPS
host. Left on with nothing listening, /api/chat fails and every healthcheck
takes ~3s longer, because the health handler probes Ollama with a 3s timeout.
Enable it by pointing OLLAMA_BASE_URL at something the container can reach.
DB_NAME is stated explicitly rather than relying on settings.py's default,
which is what the development file was leaning on.
Verified booting from this file alone, with no environment variables: binds
3000 and 8000, mounts MCP, correct password returns a token, and both the
any-password bypass and the old development password return 401.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
At the repo owner's instruction, to stop the deploy depending on re-entering
config in the Dokploy UI - which is how the container ended up exiting at
startup on missing AUTH_SECRET_KEY and returning Bad Gateway.
The two database secrets are deliberately NOT in the file. settings.py calls
load_dotenv() without override=True, so a real environment variable wins over
the file; DB_HOST and DB_PASSWORD are set in Dokploy and never enter git. Two
fields to fill instead of seven.
The auth secrets ARE committed, which is worth being explicit about:
AUTH_SECRET_KEY signs every access token, so anyone with read access to this
repository can mint a valid admin token, and git history retains it after any
rotation. .gitignore records the same warning next to the exception that allows
the file. Regenerate with scripts/make_auth_secrets.py and redeploy if that
stops being an acceptable trade.
The generated sign-in passwords are written to SIGNIN_PASSWORDS.txt, which
stays ignored - only the PBKDF2 digests are in .env, and those cannot be
reversed.
Verified end to end: the app boots on 3000 and 8000 with DB_HOST/DB_PASSWORD
supplied as environment variables, and a login with the generated admin
password returns a token while a wrong password returns 401.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
