Compare commits
6 Commits
v0.5.1-dem
...
v0.5.7-dem
| Author | SHA1 | Date | |
|---|---|---|---|
| 0558344dc2 | |||
| 8f07dee048 | |||
| 3cddd9c2e1 | |||
| 248025cdf9 | |||
| ff4f95c3b0 | |||
| 48a30d97db |
340
CLAUDE.md
340
CLAUDE.md
@@ -3359,3 +3359,343 @@ live one.
|
|||||||
- **With no engine AND nobody signed in, the engine error is still the answer.**
|
- **With no engine AND nobody signed in, the engine error is still the answer.**
|
||||||
There is nothing else to show and the person is most likely setting this PC
|
There is nothing else to show and the person is most likely setting this PC
|
||||||
up; naming head office there points them at a step they have not reached.
|
up; naming head office there points them at a step they have not reached.
|
||||||
|
|
||||||
|
## Watching a camera from the app, in another building
|
||||||
|
|
||||||
|
Snapshots answer *"is that camera working"*. They do not answer *"what is
|
||||||
|
happening in my shop right now"*, which is what somebody who opens the app
|
||||||
|
away from the counter is asking. Head office's browser already had the answer
|
||||||
|
— `LiveHub` plus `cameras.Live`, where the shop PC asks outbound whether
|
||||||
|
anybody is watching and pushes JPEG frames up for exactly as long as somebody
|
||||||
|
is — and the app could not reach it.
|
||||||
|
|
||||||
|
`cloud.CameraLive` opens that feed and the app's own loopback relay re-emits
|
||||||
|
it as **multipart MJPEG**, which is the whole trick: frames arrive base64 over
|
||||||
|
SSE, an `<img>` cannot render that, and an `<img>` renders MJPEG natively. So a
|
||||||
|
tile is an ordinary `<img>` pointed at loopback whether the camera is in this
|
||||||
|
room or another city, and no screen has to know which.
|
||||||
|
|
||||||
|
- **Reconnecting happens in the relay, not the page.** The server caps one push
|
||||||
|
at five minutes so a tab left open for a week cannot leave a shop uploading
|
||||||
|
for a week. Doing it here means the `<img>` never sees the stream end.
|
||||||
|
- **The headers are flushed before the first frame.** Go writes them on the
|
||||||
|
first body write, so without that the whole response — status line included —
|
||||||
|
waits for the shop PC to start pushing. Measured against production: thirty
|
||||||
|
seconds and not even a `Content-Type`, which surfaces as the *request* timing
|
||||||
|
out rather than a stream that has not painted yet.
|
||||||
|
- **One camera at a time.** Watching makes a shop PC upload, so a grid that
|
||||||
|
went live at once would put an estate's worth of cameras on the wire because
|
||||||
|
somebody opened a page. `Watch live` is per tile and toggles the previous one
|
||||||
|
off.
|
||||||
|
- **`live.mjpeg` is behind the same per-run token as the engine routes**, and a
|
||||||
|
wrong token is a 404 that never reaches head office at all. It is a live view
|
||||||
|
of a shop floor; the relay being on loopback is not on its own a control.
|
||||||
|
- **`CameraLive` uses its own HTTP client.** The shared one has a 30-second
|
||||||
|
timeout that covers the whole response and would therefore sever a working
|
||||||
|
live view every thirty seconds — the same trap that made the server set
|
||||||
|
`WriteTimeout` to zero for its own SSE endpoint.
|
||||||
|
|
||||||
|
## A camera read "Connected" for 34 minutes after the shop PC went blind
|
||||||
|
|
||||||
|
Found while verifying the live view against production, and it is the reason
|
||||||
|
that verification looked like a failure: head office registered the viewer and
|
||||||
|
no frame ever came.
|
||||||
|
|
||||||
|
`reportWith` returns early when the engine is unreachable — correctly, because
|
||||||
|
it has nothing to say — so the last state it sent **stays in the database
|
||||||
|
looking current**. Measured on the live estate: `cam2` and `entrance` both
|
||||||
|
reading **Connected**, in green, with `last_seen_at` thirty-four minutes old,
|
||||||
|
while the heartbeat from the same PC said `cameras_up: 0, cameras_total: 0`.
|
||||||
|
Two surfaces reading two stored fields and disagreeing about one fact.
|
||||||
|
|
||||||
|
`false` could not be the answer. It means *"this camera is not connecting"*,
|
||||||
|
which sends an installer to check cabling on a camera that was working
|
||||||
|
perfectly the last time anybody could ask it. So there are four states, not
|
||||||
|
three, and `api.CameraState` is the one function that decides them:
|
||||||
|
|
||||||
|
| state | meaning | what to do |
|
||||||
|
|---|---|---|
|
||||||
|
| `connected` | reported within `CameraStaleAfter`, and working | — |
|
||||||
|
| `not_connecting` | reported recently, and the stream will not open | check the address, password, cabling |
|
||||||
|
| `waiting` | no shop PC has ever reported this camera | it has not reached the PC yet |
|
||||||
|
| `stale` | reported once, and not lately | check the PC is on and Behavision is running |
|
||||||
|
|
||||||
|
- **`Connected` is CLEARED when the state is `stale` or `waiting`.** Leaving a
|
||||||
|
stale `true` in place keeps the lie available to every client that reads the
|
||||||
|
field directly — a mobile app, a script, an older desktop build — and leaves
|
||||||
|
two fields on one object disagreeing, which is exactly how the shops screen
|
||||||
|
once came out labelled **Working**, in green, above *"2 of 3 cameras not
|
||||||
|
connecting"*.
|
||||||
|
- **It is computed in `scanCamera`**, so every camera anybody reads passes
|
||||||
|
through it. A state computed per handler is a state one handler forgets, and
|
||||||
|
this one had already reached three screens.
|
||||||
|
- **`CameraStaleAfter` is 5 minutes — five missed reports, not one.** The agent
|
||||||
|
reports on a 60-second tick, so one miss is a dropped packet. Same reasoning
|
||||||
|
as a site being offline after three missed heartbeats: an indicator that
|
||||||
|
cries wolf is one people learn to ignore.
|
||||||
|
- **An unparseable `last_seen_at` is stale**, not connected. It should be
|
||||||
|
impossible, which is precisely why it must not fall through to the state that
|
||||||
|
says everything is fine.
|
||||||
|
|
||||||
|
## A demo on somebody else's Mac found four things, all of them silent
|
||||||
|
|
||||||
|
Three failures in one afternoon on a colleague's machine, plus one the fixing
|
||||||
|
uncovered. Every one produced a message that was true and useless.
|
||||||
|
|
||||||
|
### behavision-setup chose the Python least likely to work
|
||||||
|
|
||||||
|
`findPython` walked `3.14, 3.13, 3.12, 3.11, 3.10` and took the first hit — a
|
||||||
|
floor with **no ceiling**, which is exactly backwards. The newest Python on a
|
||||||
|
machine is the one least likely to have binary wheels for anything. It picked
|
||||||
|
3.14, pip found no numpy wheel for cp314 (`numpy<2.0` caps the resolver at
|
||||||
|
1.26.4, whose newest is cp312), fell back to building numpy from source and
|
||||||
|
produced `ERROR: Unknown compiler(s)`; once the operator had installed Xcode's
|
||||||
|
command line tools to get past that, ten minutes of compiling ended in
|
||||||
|
`<arm_neon.h> is intended only for ARM and AArch64 targets`.
|
||||||
|
|
||||||
|
Two screens of C compiler output on a shop counter, for a version choice this
|
||||||
|
program made silently. `maxMinor` refuses in one line before anything is
|
||||||
|
downloaded, and **"too new" is a different message from "too old"** — telling
|
||||||
|
somebody holding Python 3.14 that no Python was found sends them to install a
|
||||||
|
newer one, which is the direction that just failed. It is a *wheel-availability*
|
||||||
|
ceiling, not a language one: onnxruntime is the binding dependency today
|
||||||
|
(cp314 is its newest), numpy publishes further ahead, and opencv ships a
|
||||||
|
stable-ABI wheel that covers everything.
|
||||||
|
|
||||||
|
### `numpy<2.0` was the cap; OpenCV was the hazard
|
||||||
|
|
||||||
|
Widening to `<3.0` needed proof, and the proof found something else. Nine runs
|
||||||
|
of the detector guard per combination, one machine, one sitting:
|
||||||
|
|
||||||
|
```
|
||||||
|
numpy 1.26 / cv2 4.11 9 passed, 0 crashed
|
||||||
|
numpy 2.0 / cv2 4.11 8 passed, 1 crashed
|
||||||
|
numpy 1.26 / cv2 4.14 3 passed, 6 crashed
|
||||||
|
numpy 2.0 / cv2 4.14 2 passed, 7 crashed
|
||||||
|
```
|
||||||
|
|
||||||
|
**numpy is not the variable; OpenCV is** — the third row is numpy 1.26. The
|
||||||
|
crash was `test_a_shared_detector_really_does_race`, which races a shared
|
||||||
|
`cv2.FaceDetectorYN` on purpose to prove the per-camera rule. That is undefined
|
||||||
|
behaviour in C++: 4.11 usually turned it into an exception, 4.14 usually turns
|
||||||
|
it into a **segfault**, and 4.11 crashing once says the hazard was always there
|
||||||
|
and 4.11 merely survived it.
|
||||||
|
|
||||||
|
It never reached the product — `Engine._build_worker` builds a detector per
|
||||||
|
camera, which is the rule and is what the second test guards. What it reached
|
||||||
|
was the suite: two runs in three died with **no failing assertion in them**,
|
||||||
|
turning "we upgraded OpenCV" into the hardest kind of CI failure to read. The
|
||||||
|
race now runs in a **subprocess**, so a segfault is an observed outcome rather
|
||||||
|
than the end of the run, and one clean attempt proves nothing — the premise
|
||||||
|
holds if *any* of several attempts misbehaves. With that fixed the suite is
|
||||||
|
226 passed / 2 skipped on numpy 2.0.2, five runs out of five.
|
||||||
|
|
||||||
|
`opencv-python` stays capped below 5. Everything above was measured on 4.x, and
|
||||||
|
an uncapped `>=4.8.1` means every NEW install silently gets a major release
|
||||||
|
this project has never run a real camera through while every existing one keeps
|
||||||
|
4.11.
|
||||||
|
|
||||||
|
### And the fix was defeated by the wreckage of the bug
|
||||||
|
|
||||||
|
`makeVenv` reused any environment already on disk, whatever Python built it.
|
||||||
|
That machine had a runtime built by **3.14**, left behind by the run that
|
||||||
|
failed — so with the ceiling in place setup would choose a good interpreter,
|
||||||
|
reach `makeVenv`, find the 3.14 environment, keep it, and die in the same clang
|
||||||
|
error as before. A fix a user cannot reach because the bug's own debris is in
|
||||||
|
the way is not a fix, and it would have read as the release not working.
|
||||||
|
|
||||||
|
It now asks the interpreter inside an existing environment what it is and
|
||||||
|
rebuilds when the answer is unsupported, saying so. Rebuilding costs a
|
||||||
|
re-download of the libraries and nothing else — the models live in the state
|
||||||
|
root, not in there. An environment that cannot be asked counts as unusable
|
||||||
|
too: a half-created one answers nothing, and reusing it fails later in pip
|
||||||
|
with an error about a package rather than about the environment.
|
||||||
|
|
||||||
|
### One MQTT client id for a whole shop, so two PCs fought over it
|
||||||
|
|
||||||
|
`behavision-<client>-<site>` is the same string on every computer claimed to
|
||||||
|
one site. MQTT requires client ids to be unique and a broker enforces it by
|
||||||
|
disconnecting the older session when a new one arrives with the same id, so the
|
||||||
|
colleague's Mac and the shop's own till took turns kicking each other off:
|
||||||
|
|
||||||
|
```
|
||||||
|
broker connected / broker connection lost: EOF / broker connected / EOF / ...
|
||||||
|
```
|
||||||
|
|
||||||
|
**The damage is not confined to the new machine.** The shop's till is the other
|
||||||
|
half of that loop, so somebody signing in on a laptop to look at the product
|
||||||
|
stops a live shop delivering visits — and from each end it reads as an unstable
|
||||||
|
network, because nothing says otherwise.
|
||||||
|
|
||||||
|
`Config.MQTTClientID()` appends a per-installation id, minted on first load and
|
||||||
|
written back so an existing install gets one without anybody doing anything.
|
||||||
|
The site stays in the name because that is what a broker log is read *by*. A
|
||||||
|
config that could not be written falls back to a per-run id rather than a
|
||||||
|
shared one: the right failure is a new name in the log after a restart, not the
|
||||||
|
collision this exists to end.
|
||||||
|
|
||||||
|
### "no such file or directory" for an engine nobody had installed
|
||||||
|
|
||||||
|
Pressing Start with no engine went straight to the supervisor, which reported
|
||||||
|
what `exec` reported:
|
||||||
|
|
||||||
|
```
|
||||||
|
engine failed to start: fork/exec /private/var/folders/c2/.../AppTranslocation/
|
||||||
|
500A5354-.../d/Behavision.app/Contents/MacOS/engine/behavision:
|
||||||
|
no such file or directory
|
||||||
|
```
|
||||||
|
|
||||||
|
Every word true, none of it saying *run the setup tool*. The startup path did
|
||||||
|
have that sentence — in a log file nobody on a shop counter opens.
|
||||||
|
`App.engineMissing()` is now the one function the startup path, the Start
|
||||||
|
button and the status panel all consult, so three surfaces cannot give three
|
||||||
|
accounts of one fact.
|
||||||
|
|
||||||
|
It also names **App Translocation**, which is in that path and is unguessable.
|
||||||
|
macOS quarantines a downloaded app it cannot verify and runs it from a randomly
|
||||||
|
named read-only copy, so every relative path resolves inside that copy — which
|
||||||
|
is why the engine folder appears missing from a bundle that plainly contains
|
||||||
|
one, and why installing into it would not survive a restart. Fixed by dragging
|
||||||
|
the app to Applications; saying nothing leaves somebody re-running a setup tool
|
||||||
|
that cannot win. The product is unsigned, so this is the *normal* first-run
|
||||||
|
state on every Mac, not an edge case.
|
||||||
|
|
||||||
|
### And then it could not download a 230 KB file
|
||||||
|
|
||||||
|
With all of the above fixed the install succeeded on that Mac - Python 3.14
|
||||||
|
chosen and accepted, numpy 2.5.3, onnxruntime 1.30, faiss 1.15.1, the engine
|
||||||
|
itself - and setup died on the last step, fetching the YuNet model:
|
||||||
|
|
||||||
|
```
|
||||||
|
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
|
||||||
|
certificate verify failed: unable to get local issuer certificate
|
||||||
|
```
|
||||||
|
|
||||||
|
A python.org macOS build ships its **own OpenSSL with no trust store**, and
|
||||||
|
populates one only when somebody double-clicks `Install Certificates.command`
|
||||||
|
in the Python folder. Nobody installing face-recognition software has any
|
||||||
|
reason to know that exists, and the failure is forty lines of traceback about
|
||||||
|
`_ssl.c` at the end of a ten-minute install.
|
||||||
|
|
||||||
|
`_urlopen` tries the default context first and retries with **certifi's**
|
||||||
|
bundle on a verification failure. The order is the whole design:
|
||||||
|
|
||||||
|
- Default first, because on Windows and on a system or Homebrew Python the
|
||||||
|
default context reads the machine's own certificate store - which is what
|
||||||
|
makes a corporate proxy with its own root CA work. Replacing it
|
||||||
|
unconditionally would break every site that has one in order to fix a
|
||||||
|
different platform.
|
||||||
|
- certifi second, because it is already installed: `requests` is a hard
|
||||||
|
dependency and brings it.
|
||||||
|
- A `URLError` is re-raised untouched. "No route to host" and "no trust store"
|
||||||
|
are different problems, and retrying the first with a different CA list only
|
||||||
|
delays the real message.
|
||||||
|
|
||||||
|
`urlretrieve` had to go, since it offers no way to pass a context - and that is
|
||||||
|
exactly the kind of rewrite that silently drops something. The
|
||||||
|
`download: <label> <n>%` lines are a **contract**: `supervisor.go`'s
|
||||||
|
`progressRe` parses them to put first-run progress in the tray, because the API
|
||||||
|
is not up yet and a shop PC showing a stopped engine for five minutes after
|
||||||
|
install looks broken. `tests/test_model_download.py` asserts them, and the
|
||||||
|
rewritten fetch was checked against the real URL: 232,589 bytes, sha256
|
||||||
|
identical to the model already on disk.
|
||||||
|
|
||||||
|
## The engine stopped updating, and said "installed" every time
|
||||||
|
|
||||||
|
The SSL fix above was released and **did not reach the machine it was written
|
||||||
|
for**. Its log said so, one line above the tick:
|
||||||
|
|
||||||
|
```
|
||||||
|
behavision is already installed with the same version as the provided wheel.
|
||||||
|
Use --force-reinstall to force an installation of the wheel.
|
||||||
|
[ok] Engine and dependencies installed
|
||||||
|
```
|
||||||
|
|
||||||
|
and the traceback that followed still pointed at `model_assets.py", line 59,
|
||||||
|
in _fetch / urllib.request.urlretrieve` — code the fix had deleted.
|
||||||
|
|
||||||
|
Two frozen literals caused it: `version = "1.1.0"` in `pyproject.toml` and
|
||||||
|
`__version__ = "1.0.0"` in `behavision/__init__.py`. They disagreed with each
|
||||||
|
other and neither tracked a release, so **every release built
|
||||||
|
`behavision-1.1.0-py3-none-any.whl`** and `pip install --upgrade` on a machine
|
||||||
|
that already had 1.1.0 is a no-op. The comment beside that call even claimed
|
||||||
|
the opposite: *"`--upgrade` so re-running after a new release replaces the
|
||||||
|
engine rather than leaving the old one in place and reporting success."*
|
||||||
|
|
||||||
|
The shape of the damage is what makes it bad. The Go binaries — the app, the
|
||||||
|
agent, the setup tool — are rebuilt every release and updated normally. So a
|
||||||
|
shop PC ran a current app supervising an engine several releases old, and
|
||||||
|
nothing anywhere said which: `/api/health` reported the model, the paths, the
|
||||||
|
cameras and the gallery, and **no version at all**.
|
||||||
|
|
||||||
|
Three changes, and the second is the one that does not depend on the first
|
||||||
|
being remembered:
|
||||||
|
|
||||||
|
- **One version, in the package**, read by `pyproject.toml` through
|
||||||
|
`[tool.setuptools.dynamic]`. Two literals in two files is how they came to
|
||||||
|
disagree. In a checkout it reads `0.0.0+dev`, because a plausible-looking
|
||||||
|
number on a developer's `/api/health` would be worse than none.
|
||||||
|
- **`pip install --force-reinstall --no-deps <wheel>`, after the ordinary
|
||||||
|
`--upgrade`.** `--upgrade` settles the dependencies; the second call
|
||||||
|
guarantees our own code is the code in the folder. `--no-deps` is what keeps
|
||||||
|
it cheap — forcing the dependencies too would re-download ~300 MB of numpy,
|
||||||
|
OpenCV and onnxruntime on every run. A rebuild at an unchanged version is the
|
||||||
|
ordinary case while developing, so this must not rely on the version moving.
|
||||||
|
- **`release.sh` stamps the tag**: `v0.5.6-demo` → `0.5.6+demo`, valid PEP 440
|
||||||
|
(a local segment takes alphanumerics and dots, never hyphens). Into a copy of
|
||||||
|
the line, reverted in a `trap`, so the tree is never left dirty.
|
||||||
|
|
||||||
|
`/api/health` reports `version` now. Without it there is no way to tell a shop
|
||||||
|
PC three releases behind from a current one, which is precisely how this
|
||||||
|
survived several releases.
|
||||||
|
|
||||||
|
Reproduced end to end before fixing, against real wheels on Python 3.12: two
|
||||||
|
builds of the same version, `--upgrade` leaves the old code in place and prints
|
||||||
|
the same sentence the colleague's Mac printed, `--force-reinstall --no-deps`
|
||||||
|
replaces it.
|
||||||
|
|
||||||
|
### `urllib` does not let `SSLCertVerificationError` out, and a fake said it did
|
||||||
|
|
||||||
|
The certificate fix above shipped and **failed on the machine it was written
|
||||||
|
for, with the exact traceback it was meant to prevent**. The retry was written
|
||||||
|
|
||||||
|
```python
|
||||||
|
except ssl.SSLCertVerificationError:
|
||||||
|
```
|
||||||
|
|
||||||
|
and urllib never raises that from `urlopen`. It catches it and re-raises
|
||||||
|
`urllib.error.URLError(err)`, carrying the original on `.reason`. So the
|
||||||
|
`except` matched nothing, ever, and the fallback could not fire.
|
||||||
|
|
||||||
|
The unit test passed throughout, because the stub it used raised the bare SSL
|
||||||
|
error — **a shape real urllib never produces**. That is the whole lesson: a
|
||||||
|
fake that agrees with the author is worse than no test, because it converts an
|
||||||
|
untested path into a tested-looking one. The same sentence is already in this
|
||||||
|
file about `UPDATE ... RETURNING` and about the in-memory API fake, and it was
|
||||||
|
written again here anyway.
|
||||||
|
|
||||||
|
`_is_cert_failure` checks the exception **and** its `.reason`, and the tests
|
||||||
|
now raise `URLError(SSLCertVerificationError(...))`, which is what his
|
||||||
|
traceback shows. A plain `URLError` — "no route to host" — is re-raised
|
||||||
|
untouched: retrying that with a different CA list changes nothing except how
|
||||||
|
long the operator waits for the real message, and a test asserts the second
|
||||||
|
attempt never happens.
|
||||||
|
|
||||||
|
Beside the stubs there is now a **real** reproduction,
|
||||||
|
`test_against_a_real_machine_with_no_trust_store`, opt-in behind
|
||||||
|
`BEHAVISION_NETWORK_TESTS=1` because it reaches github.com. Python's default
|
||||||
|
context honours `SSL_CERT_FILE`, so pointing it at an empty file gives a
|
||||||
|
default context that trusts nobody — the python.org condition exactly — while
|
||||||
|
certifi is loaded by path and is unaffected.
|
||||||
|
|
||||||
|
It skips rather than passes where it cannot reproduce that, and the difference
|
||||||
|
is measured rather than assumed:
|
||||||
|
|
||||||
|
```
|
||||||
|
macOS Command Line Tools LibreSSL 2.8.3 128 CAs with SSL_CERT_FILE empty
|
||||||
|
(reads the system keychain)
|
||||||
|
python.org / pyenv build OpenSSL 3.5.8 0 CAs -> reproduces it
|
||||||
|
```
|
||||||
|
|
||||||
|
Checked for teeth by putting the shipped `except` back: both the corrected stub
|
||||||
|
test and the live one fail, and pass again when it is restored.
|
||||||
|
|||||||
@@ -46,6 +46,61 @@ import (
|
|||||||
// otherwise arrive as a syntax error deep inside a dependency.
|
// otherwise arrive as a syntax error deep inside a dependency.
|
||||||
const minMinor = 10
|
const minMinor = 10
|
||||||
|
|
||||||
|
// maxMinor is a WHEEL-availability ceiling, not a language one, and it is the
|
||||||
|
// reason this constant exists at all.
|
||||||
|
//
|
||||||
|
// findPython used to take the newest interpreter it could find, with a floor
|
||||||
|
// and no ceiling - which is precisely backwards, because the newest Python is
|
||||||
|
// the one least likely to have binary wheels for anything. Measured on a
|
||||||
|
// second Mac: it chose Python 3.14, pip found no numpy wheel for cp314, fell
|
||||||
|
// back to building numpy from source, and produced
|
||||||
|
//
|
||||||
|
// ERROR: Unknown compiler(s): [['cc'], ['gcc'], ['clang'], ...]
|
||||||
|
//
|
||||||
|
// then, once the operator installed Xcode's command line tools to get past
|
||||||
|
// that, ten minutes of compiling ending in
|
||||||
|
//
|
||||||
|
// arm_neon.h:28:2: error: "<arm_neon.h> is intended only for ARM and
|
||||||
|
// AArch64 targets"
|
||||||
|
//
|
||||||
|
// Two screens of C compiler output, on a shop counter, for a version choice
|
||||||
|
// made silently by this program. Refusing in one line, before anything is
|
||||||
|
// downloaded, is the whole of the fix.
|
||||||
|
//
|
||||||
|
// Raise it when the dependency set has wheels for the next version. Today
|
||||||
|
// onnxruntime is the binding one (cp314 is its newest); numpy publishes
|
||||||
|
// further ahead, and opencv-python ships a stable-ABI wheel that covers
|
||||||
|
// everything. `pip download --only-binary=:all: -r requirements.txt` against
|
||||||
|
// a candidate interpreter is the check.
|
||||||
|
const maxMinor = 14
|
||||||
|
|
||||||
|
// The three answers a candidate interpreter can get. Three, not two: a
|
||||||
|
// version that is too new and one that is too old need opposite actions from
|
||||||
|
// the operator, and collapsing them tells somebody holding Python 3.14 to go
|
||||||
|
// and install a newer Python.
|
||||||
|
const (
|
||||||
|
verdictOK = "ok"
|
||||||
|
verdictTooOld = "old"
|
||||||
|
verdictTooNew = "new"
|
||||||
|
verdictUnknown = "unparseable"
|
||||||
|
)
|
||||||
|
|
||||||
|
func pythonVerdict(major, minor int, parsed bool) string {
|
||||||
|
switch {
|
||||||
|
case !parsed:
|
||||||
|
return verdictUnknown
|
||||||
|
case major != 3:
|
||||||
|
// Python 4 is not a version this has been tried against, and 2 is
|
||||||
|
// long gone. Neither is a thing to guess about.
|
||||||
|
return verdictTooNew
|
||||||
|
case minor < minMinor:
|
||||||
|
return verdictTooOld
|
||||||
|
case minor > maxMinor:
|
||||||
|
return verdictTooNew
|
||||||
|
}
|
||||||
|
return verdictOK
|
||||||
|
}
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
if err := run(); err != nil {
|
if err := run(); err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "\n Setup did not finish: %v\n\n", err)
|
fmt.Fprintf(os.Stderr, "\n Setup did not finish: %v\n\n", err)
|
||||||
@@ -267,7 +322,13 @@ func findPython() (string, string, error) {
|
|||||||
// `python3` therefore told a Mac with Python 3.12 sitting on it to go and
|
// `python3` therefore told a Mac with Python 3.12 sitting on it to go and
|
||||||
// install Python - measured on this machine, which has 3.12 under
|
// install Python - measured on this machine, which has 3.12 under
|
||||||
// ~/.local/opt and reported "Found, but too old: python3 3.9".
|
// ~/.local/opt and reported "Found, but too old: python3 3.9".
|
||||||
versions := []string{"3.14", "3.13", "3.12", "3.11", "3.10"}
|
// Newest first WITHIN the supported range. Newest overall is what broke
|
||||||
|
// this; a version nobody has built wheels for is not a better choice than
|
||||||
|
// one that works.
|
||||||
|
var versions []string
|
||||||
|
for v := maxMinor; v >= minMinor; v-- {
|
||||||
|
versions = append(versions, fmt.Sprintf("3.%d", v))
|
||||||
|
}
|
||||||
for _, v := range versions {
|
for _, v := range versions {
|
||||||
cands = append(cands, cand{"python" + v, nil})
|
cands = append(cands, cand{"python" + v, nil})
|
||||||
}
|
}
|
||||||
@@ -290,7 +351,7 @@ func findPython() (string, string, error) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var tried []string
|
var tried, tooNew []string
|
||||||
for _, c := range cands {
|
for _, c := range cands {
|
||||||
exe := c.exe
|
exe := c.exe
|
||||||
if filepath.IsAbs(exe) {
|
if filepath.IsAbs(exe) {
|
||||||
@@ -314,7 +375,18 @@ func findPython() (string, string, error) {
|
|||||||
}
|
}
|
||||||
ver := strings.TrimSpace(string(out))
|
ver := strings.TrimSpace(string(out))
|
||||||
tried = append(tried, c.exe+" "+ver)
|
tried = append(tried, c.exe+" "+ver)
|
||||||
if major, minor, ok := parseVer(ver); ok && (major > 3 || (major == 3 && minor >= minMinor)) {
|
major, minor, parsed := parseVer(ver)
|
||||||
|
switch verdict := pythonVerdict(major, minor, parsed); verdict {
|
||||||
|
case verdictTooNew:
|
||||||
|
// Recorded separately: "too new" and "too old" need opposite
|
||||||
|
// actions, and a single "found, but unsuitable" list sends
|
||||||
|
// somebody to upgrade a Python that is already past the problem.
|
||||||
|
tooNew = append(tooNew, c.exe+" "+ver)
|
||||||
|
continue
|
||||||
|
case verdictTooOld, verdictUnknown:
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
{
|
||||||
full := exe
|
full := exe
|
||||||
if len(c.args) > 0 {
|
if len(c.args) > 0 {
|
||||||
full = exe + " " + strings.Join(c.args, " ")
|
full = exe + " " + strings.Join(c.args, " ")
|
||||||
@@ -327,7 +399,30 @@ func findPython() (string, string, error) {
|
|||||||
// python.exe to PATH" on a Windows installer page reads as software that
|
// python.exe to PATH" on a Windows installer page reads as software that
|
||||||
// does not know where it is running, which is exactly the moment somebody
|
// does not know where it is running, which is exactly the moment somebody
|
||||||
// stops trusting the rest of what it says.
|
// stops trusting the rest of what it says.
|
||||||
msg := "no Python 3.10 or newer was found on this computer.\n\n"
|
// Only a too-new Python is a different problem with a different fix, and
|
||||||
|
// saying "no Python was found" to somebody looking at Python 3.14 is the
|
||||||
|
// kind of message that makes people stop believing the next one.
|
||||||
|
if len(tooNew) > 0 && len(tried) == 0 {
|
||||||
|
// Built as a value and wrapped, not written as an fmt.Errorf literal:
|
||||||
|
// this is a paragraph shown to an operator, and a linter that wants
|
||||||
|
// error strings to be lower-case fragments is right about errors
|
||||||
|
// programs read and wrong about the ones people do.
|
||||||
|
tooNewMsg := fmt.Sprintf(
|
||||||
|
"this computer has %s, which is newer than Behavision supports.\n\n"+
|
||||||
|
" Some of the libraries the engine needs have no build for it\n"+
|
||||||
|
" yet, so installing would fail part-way through.\n\n"+
|
||||||
|
" Install Python 3.%d and run this again:\n"+
|
||||||
|
" macOS: brew install python@3.%d\n"+
|
||||||
|
" or https://www.python.org/downloads/macos/\n"+
|
||||||
|
" Windows: https://www.python.org/downloads/windows/\n\n"+
|
||||||
|
" Both versions can sit on the machine together; this picks\n"+
|
||||||
|
" the one it can use.",
|
||||||
|
strings.Join(tooNew, ", "), maxMinor, maxMinor)
|
||||||
|
return "", "", errors.New(tooNewMsg)
|
||||||
|
}
|
||||||
|
|
||||||
|
msg := fmt.Sprintf("no Python between 3.%d and 3.%d was found on this computer.\n\n",
|
||||||
|
minMinor, maxMinor)
|
||||||
if runtime.GOOS == "windows" {
|
if runtime.GOOS == "windows" {
|
||||||
msg += " Install it from https://www.python.org/downloads/windows/\n" +
|
msg += " Install it from https://www.python.org/downloads/windows/\n" +
|
||||||
" and tick \"Add python.exe to PATH\" on the first screen,\n" +
|
" and tick \"Add python.exe to PATH\" on the first screen,\n" +
|
||||||
@@ -339,6 +434,9 @@ func findPython() (string, string, error) {
|
|||||||
if len(tried) > 0 {
|
if len(tried) > 0 {
|
||||||
msg += "\n\n Found, but too old: " + strings.Join(tried, ", ")
|
msg += "\n\n Found, but too old: " + strings.Join(tried, ", ")
|
||||||
}
|
}
|
||||||
|
if len(tooNew) > 0 {
|
||||||
|
msg += "\n\n Found, but too new: " + strings.Join(tooNew, ", ")
|
||||||
|
}
|
||||||
return "", "", errors.New(msg)
|
return "", "", errors.New(msg)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -375,14 +473,52 @@ func venvPython(venv string) string {
|
|||||||
// engine requires - inside a shared interpreter is how you break the other
|
// engine requires - inside a shared interpreter is how you break the other
|
||||||
// thing months later, silently.
|
// thing months later, silently.
|
||||||
func makeVenv(py, venv string) error {
|
func makeVenv(py, venv string) error {
|
||||||
|
// An existing environment is reused - but only if the Python inside it is
|
||||||
|
// one this build supports.
|
||||||
|
//
|
||||||
|
// It used to be reused unconditionally, and that would have made the
|
||||||
|
// version ceiling above look like it did not work. The machine this was
|
||||||
|
// all found on already had a runtime built by Python 3.14, from the run
|
||||||
|
// that failed: with the ceiling in place setup would choose a good
|
||||||
|
// interpreter, reach here, find the 3.14 environment, keep it, and die in
|
||||||
|
// the same clang error as before. A fix that is defeated by the wreckage
|
||||||
|
// of the bug it fixes is not one.
|
||||||
|
//
|
||||||
|
// Rebuilding costs a re-download of the libraries and nothing else. The
|
||||||
|
// models are in the state root, not in here, so they survive.
|
||||||
if _, err := os.Stat(venvPython(venv)); err == nil {
|
if _, err := os.Stat(venvPython(venv)); err == nil {
|
||||||
return nil // already built; pip below brings it up to date
|
ok, ver := venvUsable(venv)
|
||||||
|
if ok {
|
||||||
|
return nil // pip below brings it up to date
|
||||||
|
}
|
||||||
|
fmt.Printf(" [..] %-24s %s\n", "Rebuilding environment",
|
||||||
|
"the existing one uses "+ver+", which is not supported")
|
||||||
|
if err := os.RemoveAll(venv); err != nil {
|
||||||
|
return fmt.Errorf("removing the old environment at %s: %w", venv, err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
exe, args := splitLauncher(py)
|
exe, args := splitLauncher(py)
|
||||||
args = append(args, "-m", "venv", venv)
|
args = append(args, "-m", "venv", venv)
|
||||||
return stream(exec.Command(exe, args...), "creating the virtual environment")
|
return stream(exec.Command(exe, args...), "creating the virtual environment")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// venvUsable reports whether the interpreter already inside an environment is
|
||||||
|
// one this build supports, and what it is when it is not.
|
||||||
|
//
|
||||||
|
// An environment that cannot be asked counts as unusable: a half-created or
|
||||||
|
// truncated one answers nothing, and reusing it fails later in pip with an
|
||||||
|
// error about a package rather than about the environment.
|
||||||
|
func venvUsable(venv string) (bool, string) {
|
||||||
|
out, err := exec.Command(venvPython(venv), "-c",
|
||||||
|
"import sys;print('%d.%d'%sys.version_info[:2])").Output()
|
||||||
|
if err != nil {
|
||||||
|
return false, "an interpreter that will not run"
|
||||||
|
}
|
||||||
|
ver := strings.TrimSpace(string(out))
|
||||||
|
major, minor, parsed := parseVer(ver)
|
||||||
|
return pythonVerdict(major, minor, parsed) == verdictOK, "Python " + ver
|
||||||
|
}
|
||||||
|
|
||||||
func pipInstall(vpy, src string) error {
|
func pipInstall(vpy, src string) error {
|
||||||
fmt.Println(" Installing the engine and its libraries. This downloads a few")
|
fmt.Println(" Installing the engine and its libraries. This downloads a few")
|
||||||
fmt.Println(" hundred megabytes and takes a while on a slow connection.")
|
fmt.Println(" hundred megabytes and takes a while on a slow connection.")
|
||||||
@@ -403,7 +539,30 @@ func pipInstall(vpy, src string) error {
|
|||||||
// 'behavision.egg-info': Read-only file system". Falling back to source
|
// 'behavision.egg-info': Read-only file system". Falling back to source
|
||||||
// copies it somewhere writable first, for the same reason.
|
// copies it somewhere writable first, for the same reason.
|
||||||
if wheels, _ := filepath.Glob(filepath.Join(src, "behavision-*.whl")); len(wheels) > 0 {
|
if wheels, _ := filepath.Glob(filepath.Join(src, "behavision-*.whl")); len(wheels) > 0 {
|
||||||
return stream(exec.Command(vpy, "-m", "pip", "install", "--upgrade", wheels[0]),
|
// TWO calls, and the second is the one that matters.
|
||||||
|
//
|
||||||
|
// `--upgrade` alone is not an upgrade when the version has not moved:
|
||||||
|
// pip skips the wheel and says so, one line above this program
|
||||||
|
// printing "[ok] Engine and dependencies installed". The wheel version
|
||||||
|
// was a frozen literal for several releases, so every engine fix in
|
||||||
|
// them silently failed to reach any machine that had run setup once -
|
||||||
|
// while the Go binaries beside it, rebuilt every release, updated
|
||||||
|
// normally. Half the product current, half of it months old, and
|
||||||
|
// nothing saying which.
|
||||||
|
//
|
||||||
|
// release.sh stamps the tag into the version now, so the versions do
|
||||||
|
// differ. This does not rely on that: a rebuild at the same version is
|
||||||
|
// the ordinary case while developing, and "installed" has to mean the
|
||||||
|
// code in this folder either way.
|
||||||
|
if err := stream(exec.Command(vpy, "-m", "pip", "install", "--upgrade", wheels[0]),
|
||||||
|
"installing the engine"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// --no-deps so this is our own package only: the call above has
|
||||||
|
// already settled the dependencies, and forcing those too would
|
||||||
|
// re-download ~300 MB of numpy, OpenCV and onnxruntime every run.
|
||||||
|
return stream(exec.Command(vpy, "-m", "pip", "install",
|
||||||
|
"--force-reinstall", "--no-deps", wheels[0]),
|
||||||
"installing the engine")
|
"installing the engine")
|
||||||
}
|
}
|
||||||
tmp, err := os.MkdirTemp("", "behavision-src-")
|
tmp, err := os.MkdirTemp("", "behavision-src-")
|
||||||
|
|||||||
112
agent/cmd/behavision-setup/python_test.go
Normal file
112
agent/cmd/behavision-setup/python_test.go
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The choice this program makes silently, and got wrong.
|
||||||
|
//
|
||||||
|
// findPython took the newest interpreter on the machine, with a floor and no
|
||||||
|
// ceiling - backwards, because the newest Python is the one least likely to
|
||||||
|
// have binary wheels. On a Mac holding Python 3.14 it chose 3.14, pip found
|
||||||
|
// no numpy wheel for cp314, fell back to a source build and produced two
|
||||||
|
// screens of clang errors ending in "<arm_neon.h> is intended only for ARM
|
||||||
|
// and AArch64 targets". The operator's machine was fine; the version was not.
|
||||||
|
func TestTooNewIsRefusedRatherThanCompiled(t *testing.T) {
|
||||||
|
if got := pythonVerdict(3, maxMinor+1, true); got != verdictTooNew {
|
||||||
|
t.Errorf("3.%d = %q, want %q - picking it means a source build",
|
||||||
|
maxMinor+1, got, verdictTooNew)
|
||||||
|
}
|
||||||
|
if got := pythonVerdict(3, maxMinor, true); got != verdictOK {
|
||||||
|
t.Errorf("3.%d = %q, want %q - the ceiling is inclusive", maxMinor, got, verdictOK)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Too old and too new must stay different answers. Telling somebody holding
|
||||||
|
// Python 3.14 that no Python was found, or that theirs is too old, sends them
|
||||||
|
// to install a newer one - which is the direction that already failed.
|
||||||
|
func TestOldAndNewAreDifferentAnswers(t *testing.T) {
|
||||||
|
old := pythonVerdict(3, minMinor-1, true)
|
||||||
|
fresh := pythonVerdict(3, maxMinor+1, true)
|
||||||
|
if old == fresh {
|
||||||
|
t.Fatalf("3.%d and 3.%d both reported %q", minMinor-1, maxMinor+1, old)
|
||||||
|
}
|
||||||
|
if old != verdictTooOld {
|
||||||
|
t.Errorf("3.%d = %q, want %q", minMinor-1, old, verdictTooOld)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every version in the range is accepted, so the window this program claims
|
||||||
|
// to support is the one it actually uses.
|
||||||
|
func TestTheWholeSupportedRangeIsAccepted(t *testing.T) {
|
||||||
|
for m := minMinor; m <= maxMinor; m++ {
|
||||||
|
if got := pythonVerdict(3, m, true); got != verdictOK {
|
||||||
|
t.Errorf("3.%d = %q, want %q", m, got, verdictOK)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if minMinor > maxMinor {
|
||||||
|
t.Fatal("the supported range is empty; nothing would ever be chosen")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A major version nobody has tested against is not something to guess at, and
|
||||||
|
// an unreadable version string is not a working interpreter.
|
||||||
|
func TestUnknownVersionsAreNotAccepted(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
name string
|
||||||
|
major, minor int
|
||||||
|
parsed bool
|
||||||
|
}{
|
||||||
|
{"python 4", 4, 0, true},
|
||||||
|
{"python 2", 2, 7, true},
|
||||||
|
{"unparseable", 0, 0, false},
|
||||||
|
} {
|
||||||
|
if got := pythonVerdict(c.major, c.minor, c.parsed); got == verdictOK {
|
||||||
|
t.Errorf("%s was accepted", c.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An environment already on disk is reused, and that is right until the Python
|
||||||
|
// inside it is one this build cannot use.
|
||||||
|
//
|
||||||
|
// It was reused unconditionally, which would have defeated the ceiling above
|
||||||
|
// on the exact machine that found the bug: that Mac already had a runtime
|
||||||
|
// built by Python 3.14, left behind by the run that failed. Setup would pick a
|
||||||
|
// good interpreter, find the 3.14 environment, keep it, and die in the same
|
||||||
|
// clang error as before - a fix defeated by the wreckage of the bug it fixes.
|
||||||
|
//
|
||||||
|
// Real environments, not a fake: the thing under test is what an interpreter
|
||||||
|
// on disk reports about itself.
|
||||||
|
func TestAnUnsupportedEnvironmentIsNotReused(t *testing.T) {
|
||||||
|
py, _, err := findPython()
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("no supported Python on this machine: %v", err)
|
||||||
|
}
|
||||||
|
venv := filepath.Join(t.TempDir(), "runtime")
|
||||||
|
if err := makeVenv(py, venv); err != nil {
|
||||||
|
t.Fatalf("makeVenv: %v", err)
|
||||||
|
}
|
||||||
|
if ok, ver := venvUsable(venv); !ok {
|
||||||
|
t.Fatalf("an environment built from the interpreter setup just chose "+
|
||||||
|
"reported itself unusable (%s)", ver)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The two states that must not be confused with a working one.
|
||||||
|
empty := filepath.Join(t.TempDir(), "gone")
|
||||||
|
if ok, _ := venvUsable(empty); ok {
|
||||||
|
t.Error("a missing environment was reported usable")
|
||||||
|
}
|
||||||
|
broken := filepath.Join(t.TempDir(), "broken")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(venvPython(broken)), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(venvPython(broken), []byte("not an interpreter"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ok, ver := venvUsable(broken); ok {
|
||||||
|
t.Errorf("a half-created environment was reported usable (%s)", ver)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -325,7 +325,7 @@ func cmdRun() error {
|
|||||||
cfg.ClientID, cfg.SiteID, cfg.BrokerURL)
|
cfg.ClientID, cfg.SiteID, cfg.BrokerURL)
|
||||||
client, err := mqtt.NewClient(mqtt.ClientOptions{
|
client, err := mqtt.NewClient(mqtt.ClientOptions{
|
||||||
BrokerURL: cfg.BrokerURL,
|
BrokerURL: cfg.BrokerURL,
|
||||||
ClientID: "behavision-" + cfg.ClientID + "-" + cfg.SiteID,
|
ClientID: cfg.MQTTClientID(),
|
||||||
Username: cfg.BrokerUsername, Password: cfg.BrokerPassword,
|
Username: cfg.BrokerUsername, Password: cfg.BrokerPassword,
|
||||||
CAFile: cfg.BrokerCAFile, Log: logger,
|
CAFile: cfg.BrokerCAFile, Log: logger,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -30,12 +30,12 @@ import (
|
|||||||
// argument, and it is why the wanted-check comes first and the push stops the
|
// argument, and it is why the wanted-check comes first and the push stops the
|
||||||
// moment the server says the last viewer has gone.
|
// moment the server says the last viewer has gone.
|
||||||
type Live struct {
|
type Live struct {
|
||||||
Engine *EngineClient
|
Engine *EngineClient
|
||||||
Cloud *CloudClient
|
Cloud *CloudClient
|
||||||
Log *log.Logger
|
Log *log.Logger
|
||||||
FPS float64
|
FPS float64
|
||||||
Width int
|
Width int
|
||||||
Quality int
|
Quality int
|
||||||
}
|
}
|
||||||
|
|
||||||
// Defaults, measured against the office camera rather than guessed.
|
// Defaults, measured against the office camera rather than guessed.
|
||||||
|
|||||||
@@ -8,7 +8,9 @@
|
|||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"crypto/rand"
|
||||||
"encoding/base64"
|
"encoding/base64"
|
||||||
|
"encoding/hex"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
@@ -83,6 +85,10 @@ type Config struct {
|
|||||||
// Queue.
|
// Queue.
|
||||||
SpoolMax int `json:"spool_max"`
|
SpoolMax int `json:"spool_max"`
|
||||||
|
|
||||||
|
// InstallID distinguishes THIS installation from every other one claimed
|
||||||
|
// to the same site. See MQTTClientID.
|
||||||
|
InstallID string `json:"install_id,omitempty"`
|
||||||
|
|
||||||
path string
|
path string
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -124,6 +130,14 @@ func Load(path string) (Config, error) {
|
|||||||
return cfg, fmt.Errorf("config %s: %w", path, err)
|
return cfg, fmt.Errorf("config %s: %w", path, err)
|
||||||
}
|
}
|
||||||
cfg.path = path
|
cfg.path = path
|
||||||
|
// Minted on first load and written back, so an installation that predates
|
||||||
|
// this field gets one without anybody doing anything. Best effort: a
|
||||||
|
// read-only config still yields a working id for this run, it is simply
|
||||||
|
// not the same one next time.
|
||||||
|
if cfg.InstallID == "" {
|
||||||
|
cfg.InstallID = newInstallID()
|
||||||
|
_ = cfg.Save(path)
|
||||||
|
}
|
||||||
for _, field := range []*string{&cfg.BrokerPassword, &cfg.APIPassword,
|
for _, field := range []*string{&cfg.BrokerPassword, &cfg.APIPassword,
|
||||||
&cfg.SessionToken, &cfg.SessionRefresh, &cfg.AgentToken} {
|
&cfg.SessionToken, &cfg.SessionRefresh, &cfg.AgentToken} {
|
||||||
plain, err := reveal(*field)
|
plain, err := reveal(*field)
|
||||||
@@ -210,3 +224,41 @@ func reveal(stored string) (string, error) {
|
|||||||
}
|
}
|
||||||
return string(plain), nil
|
return string(plain), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MQTTClientID names this INSTALLATION, not this site.
|
||||||
|
//
|
||||||
|
// It was `behavision-<client>-<site>`, which is the same string on every
|
||||||
|
// computer claimed to one shop. MQTT requires client ids to be unique and a
|
||||||
|
// broker enforces it by disconnecting the older session when a new one
|
||||||
|
// arrives with the same id - so two machines on one site take turns kicking
|
||||||
|
// each other off, forever. Measured on a second Mac claimed to a live shop:
|
||||||
|
//
|
||||||
|
// broker connected / broker connection lost: EOF / broker connected / ...
|
||||||
|
//
|
||||||
|
// The damage is not confined to the new machine. The shop's own till is the
|
||||||
|
// other half of that loop, so somebody signing in on a laptop to look at the
|
||||||
|
// product stops the shop delivering visits - and nothing at either end says
|
||||||
|
// why, because from each side it reads as an unstable network.
|
||||||
|
//
|
||||||
|
// The site stays in the id because it is what a broker log is read by, and
|
||||||
|
// the random half is short for the same reason. `CleanSession(true)` means
|
||||||
|
// there is no session state for a changed id to strand.
|
||||||
|
func (c Config) MQTTClientID() string {
|
||||||
|
id := c.InstallID
|
||||||
|
if id == "" {
|
||||||
|
// A config that could not be written still has to produce a UNIQUE
|
||||||
|
// id, or this falls straight back into the collision it exists to
|
||||||
|
// prevent. Per-run is the right failure: the connection works and the
|
||||||
|
// only cost is a new name in the broker's log after a restart.
|
||||||
|
id = newInstallID()
|
||||||
|
}
|
||||||
|
return "behavision-" + c.ClientID + "-" + c.SiteID + "-" + id
|
||||||
|
}
|
||||||
|
|
||||||
|
func newInstallID() string {
|
||||||
|
b := make([]byte, 4)
|
||||||
|
if _, err := rand.Read(b); err != nil {
|
||||||
|
return "x"
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(b)
|
||||||
|
}
|
||||||
|
|||||||
88
agent/pkg/config/installid_test.go
Normal file
88
agent/pkg/config/installid_test.go
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The bug this exists to prevent, measured on a second Mac claimed to a live
|
||||||
|
// shop: MQTT requires client ids to be unique, and a broker enforces it by
|
||||||
|
// disconnecting the older session when a new one arrives with the same id. The
|
||||||
|
// id was `behavision-<client>-<site>` - identical on every computer claimed to
|
||||||
|
// one shop - so the two took turns kicking each other off:
|
||||||
|
//
|
||||||
|
// broker connected / broker connection lost: EOF / broker connected / ...
|
||||||
|
//
|
||||||
|
// The damage is not confined to the new machine. The shop's own till is the
|
||||||
|
// other half of that loop, so somebody signing in on a laptop to look at the
|
||||||
|
// product stops the shop delivering visits.
|
||||||
|
func TestTwoInstallsOnOneSiteGetDifferentClientIDs(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
one := writeClaimed(t, filepath.Join(dir, "a.json"))
|
||||||
|
two := writeClaimed(t, filepath.Join(dir, "b.json"))
|
||||||
|
|
||||||
|
if one.MQTTClientID() == two.MQTTClientID() {
|
||||||
|
t.Fatalf("both installs answered to %q; the broker will disconnect one "+
|
||||||
|
"whenever the other connects", one.MQTTClientID())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the same install keeps its name across restarts, or a broker log is a
|
||||||
|
// list of strangers and nobody can tell one till from a stream of new ones.
|
||||||
|
func TestOneInstallKeepsItsClientIDAcrossRestarts(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "agent.json")
|
||||||
|
first := writeClaimed(t, path)
|
||||||
|
|
||||||
|
again, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("reload: %v", err)
|
||||||
|
}
|
||||||
|
if got, want := again.MQTTClientID(), first.MQTTClientID(); got != want {
|
||||||
|
t.Errorf("after a restart the id was %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The site stays in the id: it is what somebody reading a broker log is
|
||||||
|
// reading FOR, and an opaque random string would make every connection
|
||||||
|
// anonymous.
|
||||||
|
func TestTheClientIDStillNamesTheShop(t *testing.T) {
|
||||||
|
c := Config{ClientID: "tenext-retail", SiteID: "chennai", InstallID: "abcd1234"}
|
||||||
|
id := c.MQTTClientID()
|
||||||
|
for _, want := range []string{"tenext-retail", "chennai", "abcd1234"} {
|
||||||
|
if !strings.Contains(id, want) {
|
||||||
|
t.Errorf("client id %q does not contain %q", id, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A config that could not be written still has to produce a UNIQUE id, or a
|
||||||
|
// read-only install falls straight back into the collision. Per-run is the
|
||||||
|
// right failure: the connection works, and the only cost is a new name in the
|
||||||
|
// broker's log after a restart.
|
||||||
|
func TestAnUnsavedConfigStillGetsAUniqueID(t *testing.T) {
|
||||||
|
a := Config{ClientID: "c", SiteID: "s"}
|
||||||
|
b := Config{ClientID: "c", SiteID: "s"}
|
||||||
|
if a.MQTTClientID() == b.MQTTClientID() {
|
||||||
|
t.Fatal("two configs with no install id produced the same client id")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeClaimed(t *testing.T, path string) Config {
|
||||||
|
t.Helper()
|
||||||
|
cfg := Defaults()
|
||||||
|
cfg.ClientID, cfg.SiteID = "tenext-retail", "chennai"
|
||||||
|
if err := cfg.Save(path); err != nil {
|
||||||
|
t.Fatalf("save: %v", err)
|
||||||
|
}
|
||||||
|
// Loading is what mints the id, so an installation that predates the
|
||||||
|
// field gets one without anybody doing anything.
|
||||||
|
got, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load: %v", err)
|
||||||
|
}
|
||||||
|
if got.InstallID == "" {
|
||||||
|
t.Fatal("loading a config without an install id did not mint one")
|
||||||
|
}
|
||||||
|
return got
|
||||||
|
}
|
||||||
@@ -4,4 +4,15 @@ Pipeline: capture -> detect (YuNet) -> track (IoU) -> align + encode
|
|||||||
(ArcFace ONNX) -> match / auto-enroll (FAISS + SQLite) -> events + API.
|
(ArcFace ONNX) -> match / auto-enroll (FAISS + SQLite) -> events + API.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
__version__ = "1.0.0"
|
# Stamped by release.sh from the git tag, into a copy of this line, so a
|
||||||
|
# release always produces a wheel nobody has installed before. It was a frozen
|
||||||
|
# "1.0.0" here and a frozen "1.1.0" in pyproject.toml - two literals that
|
||||||
|
# disagreed with each other and tracked nothing - which is how every engine fix
|
||||||
|
# for several releases silently failed to reach a machine that had run setup
|
||||||
|
# once. pip skips a wheel whose version is already installed and says so, one
|
||||||
|
# line above setup printing "[ok] Engine and dependencies installed".
|
||||||
|
#
|
||||||
|
# In a checkout it stays obviously a checkout: "0.0.0+dev" on /api/health is
|
||||||
|
# the truth about a developer's machine, and a plausible-looking number there
|
||||||
|
# would be worse than none.
|
||||||
|
__version__ = "0.0.0+dev"
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ from fastapi.responses import HTMLResponse, Response, StreamingResponse
|
|||||||
from fastapi.security import HTTPBasic, HTTPBasicCredentials
|
from fastapi.security import HTTPBasic, HTTPBasicCredentials
|
||||||
from pydantic import BaseModel, ValidationError
|
from pydantic import BaseModel, ValidationError
|
||||||
|
|
||||||
|
from . import __version__
|
||||||
from .config import ApiSection, CameraConfig, CameraTuning
|
from .config import ApiSection, CameraConfig, CameraTuning
|
||||||
from .commission import CommissionRun
|
from .commission import CommissionRun
|
||||||
from .events import Event
|
from .events import Event
|
||||||
@@ -142,7 +143,7 @@ def _reencode(jpeg: bytes, width: int, quality: int) -> "bytes | None":
|
|||||||
|
|
||||||
|
|
||||||
def create_app(engine: Engine) -> FastAPI:
|
def create_app(engine: Engine) -> FastAPI:
|
||||||
app = FastAPI(title="Behavision", version="1.0.0",
|
app = FastAPI(title="Behavision", version=__version__,
|
||||||
dependencies=_auth_dependencies(engine.cfg.api))
|
dependencies=_auth_dependencies(engine.cfg.api))
|
||||||
|
|
||||||
def worker_or_404(camera_id: str):
|
def worker_or_404(camera_id: str):
|
||||||
@@ -172,6 +173,10 @@ def create_app(engine: Engine) -> FastAPI:
|
|||||||
# are up, and the shop recognises nobody it already knows.
|
# are up, and the shop recognises nobody it already knows.
|
||||||
stranded = engine.gallery.health["stranded"]
|
stranded = engine.gallery.health["stranded"]
|
||||||
return {"status": "ok" if engine.started_at else "starting",
|
return {"status": "ok" if engine.started_at else "starting",
|
||||||
|
# Which build is actually running. Without it there was no way
|
||||||
|
# to tell a shop PC three releases behind from a current one -
|
||||||
|
# which is precisely how a silent install failure survived.
|
||||||
|
"version": __version__,
|
||||||
"recognition_model": engine.encoder.model_name,
|
"recognition_model": engine.encoder.model_name,
|
||||||
"gallery_unreadable_embeddings": stranded,
|
"gallery_unreadable_embeddings": stranded,
|
||||||
# "where is my database" must be answerable from the API: the
|
# "where is my database" must be answerable from the API: the
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
import shutil
|
import shutil
|
||||||
|
import ssl
|
||||||
|
import urllib.error
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -35,6 +37,81 @@ _COPY_MAP = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _https_context() -> "ssl.SSLContext | None":
|
||||||
|
"""The CA store to trust, or None to use whatever Python defaults to.
|
||||||
|
|
||||||
|
Returning None first is deliberate. On Windows and on a Homebrew or
|
||||||
|
system Python, the default context reads the machine's own certificate
|
||||||
|
store - which is what makes a corporate proxy with its own root CA work.
|
||||||
|
Replacing that with certifi's bundle unconditionally would break every
|
||||||
|
site that has one, in order to fix a different platform.
|
||||||
|
|
||||||
|
The platform this fixes is a python.org macOS build. It ships its own
|
||||||
|
OpenSSL with NO trust store, and populates one only when somebody
|
||||||
|
double-clicks `Install Certificates.command` in the Python folder -
|
||||||
|
which nobody installing face-recognition software has any reason to know
|
||||||
|
about. Every HTTPS request from that interpreter fails with:
|
||||||
|
|
||||||
|
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
|
||||||
|
certificate verify failed: unable to get local issuer certificate
|
||||||
|
|
||||||
|
Measured on a colleague's Mac: the engine installed perfectly and then
|
||||||
|
could not download a 230 KB model file, ending setup in forty lines of
|
||||||
|
traceback about `_ssl.c`.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import certifi
|
||||||
|
except ImportError: # pragma: no cover - certifi ships with requests
|
||||||
|
return None
|
||||||
|
return ssl.create_default_context(cafile=certifi.where())
|
||||||
|
|
||||||
|
|
||||||
|
def _is_cert_failure(err: BaseException) -> bool:
|
||||||
|
"""Is this a certificate-verification failure, however it is wrapped?
|
||||||
|
|
||||||
|
`urllib` does NOT let `ssl.SSLCertVerificationError` out. It catches it and
|
||||||
|
re-raises `urllib.error.URLError(err)`, carrying the original on `.reason`
|
||||||
|
- so `except ssl.SSLCertVerificationError` around `urlopen` matches
|
||||||
|
nothing, ever.
|
||||||
|
|
||||||
|
That is not a subtlety this file gets to record academically: the first
|
||||||
|
version of the fallback below was written exactly that way, shipped, and
|
||||||
|
failed on the machine it was written for with the very traceback it was
|
||||||
|
meant to prevent. The unit test passed throughout, because the fake it
|
||||||
|
used raised the bare SSL error - a shape real urllib never produces. A
|
||||||
|
stub that agrees with the author is worse than no test, and the test now
|
||||||
|
raises what urllib raises.
|
||||||
|
"""
|
||||||
|
reason = getattr(err, "reason", None)
|
||||||
|
return isinstance(err, ssl.SSLCertVerificationError) or \
|
||||||
|
isinstance(reason, ssl.SSLCertVerificationError)
|
||||||
|
|
||||||
|
|
||||||
|
def _urlopen(url: str, timeout: float = 60.0):
|
||||||
|
"""Open a URL, falling back to certifi's CA bundle on a verify failure.
|
||||||
|
|
||||||
|
Default first, certifi second, so the fix is additive: a machine whose
|
||||||
|
own store works keeps using it, and one with no store at all gets a
|
||||||
|
bundle rather than a traceback. certifi is already here - `requests` is a
|
||||||
|
hard dependency and brings it.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return urllib.request.urlopen(url, timeout=timeout)
|
||||||
|
except (urllib.error.URLError, ssl.SSLCertVerificationError) as err:
|
||||||
|
# Only a certificate problem is worth a second attempt. "No route to
|
||||||
|
# host" and "connection refused" arrive as URLError too, and retrying
|
||||||
|
# those with a different CA list changes nothing except how long the
|
||||||
|
# operator waits for the real message.
|
||||||
|
if not _is_cert_failure(err):
|
||||||
|
raise
|
||||||
|
ctx = _https_context()
|
||||||
|
if ctx is None:
|
||||||
|
raise
|
||||||
|
log.info("the system certificate store could not verify %s; "
|
||||||
|
"using the bundled CA list", url.split("/")[2])
|
||||||
|
return urllib.request.urlopen(url, timeout=timeout, context=ctx)
|
||||||
|
|
||||||
|
|
||||||
def _fetch(url: str, dest: Path, label: str) -> None:
|
def _fetch(url: str, dest: Path, label: str) -> None:
|
||||||
"""Download with progress on stdout the supervisor can read.
|
"""Download with progress on stdout the supervisor can read.
|
||||||
|
|
||||||
@@ -56,7 +133,22 @@ def _fetch(url: str, dest: Path, label: str) -> None:
|
|||||||
last = pct
|
last = pct
|
||||||
log.info("download: %s %d%%", label, pct)
|
log.info("download: %s %d%%", label, pct)
|
||||||
|
|
||||||
urllib.request.urlretrieve(url, dest, hook)
|
# Streamed rather than urlretrieve, only because urlretrieve offers no way
|
||||||
|
# to pass an SSL context and the whole point here is choosing one. The
|
||||||
|
# `download: <label> <n>%` lines are a contract: the supervisor parses
|
||||||
|
# them (`progressRe`) to put first-run progress in the tray, and without
|
||||||
|
# them a shop PC shows a stopped engine for five minutes after install.
|
||||||
|
with _urlopen(url) as resp:
|
||||||
|
total = int(resp.headers.get("Content-Length") or 0)
|
||||||
|
blocks, block_size = 0, 64 * 1024
|
||||||
|
with open(dest, "wb") as out:
|
||||||
|
while True:
|
||||||
|
chunk = resp.read(block_size)
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
out.write(chunk)
|
||||||
|
blocks += 1
|
||||||
|
hook(blocks, block_size, total)
|
||||||
log.info("download: %s 100%%", label)
|
log.info("download: %s 100%%", label)
|
||||||
|
|
||||||
|
|
||||||
@@ -91,7 +183,7 @@ def setup_models(models_dir: Path) -> "list[str]":
|
|||||||
import io
|
import io
|
||||||
import zipfile
|
import zipfile
|
||||||
|
|
||||||
with urllib.request.urlopen(BUFFALO_SC_URL) as resp:
|
with _urlopen(BUFFALO_SC_URL) as resp:
|
||||||
payload = io.BytesIO(resp.read())
|
payload = io.BytesIO(resp.read())
|
||||||
with zipfile.ZipFile(payload) as zf, \
|
with zipfile.ZipFile(payload) as zf, \
|
||||||
zf.open("w600k_mbf.onnx") as src, \
|
zf.open("w600k_mbf.onnx") as src, \
|
||||||
|
|||||||
@@ -44,6 +44,9 @@ type App struct {
|
|||||||
broker *agentmqtt.Client
|
broker *agentmqtt.Client
|
||||||
stopBridge func()
|
stopBridge func()
|
||||||
hookURL string
|
hookURL string
|
||||||
|
// The resolved engine command, so engineMissing() and the supervisor are
|
||||||
|
// never looking at two different paths.
|
||||||
|
engineExe string
|
||||||
// Relays camera feeds to the webview so the engine's credential never has
|
// Relays camera feeds to the webview so the engine's credential never has
|
||||||
// to travel in an <img> src, which a Chromium webview would strip anyway.
|
// to travel in an <img> src, which a Chromium webview would strip anyway.
|
||||||
proxy *streamProxy
|
proxy *streamProxy
|
||||||
@@ -82,6 +85,14 @@ func (a *App) startup(ctx context.Context) {
|
|||||||
if err := a.proxy.start(a.local.Base, a.local.User, a.local.Password); err != nil {
|
if err := a.proxy.start(a.local.Base, a.local.User, a.local.Password); err != nil {
|
||||||
log.Printf("camera relay unavailable, tiles will not load: %v", err)
|
log.Printf("camera relay unavailable, tiles will not load: %v", err)
|
||||||
}
|
}
|
||||||
|
// And the other direction: watching a camera in another building, through
|
||||||
|
// head office's relay. Enabled unconditionally rather than only when a
|
||||||
|
// session already exists, because signing in is a thing that happens
|
||||||
|
// while the app is open - and CameraLive refuses without a session
|
||||||
|
// anyway, so there is nothing to gate.
|
||||||
|
if err := a.proxy.watchRemote(a.cloud.CameraLive); err != nil {
|
||||||
|
log.Printf("remote camera view unavailable: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
// A saved session means a shop PC that rebooted overnight comes back
|
// A saved session means a shop PC that rebooted overnight comes back
|
||||||
// working instead of waiting for someone to log in.
|
// working instead of waiting for someone to log in.
|
||||||
@@ -101,6 +112,9 @@ func (a *App) startup(ctx context.Context) {
|
|||||||
if exe != "" && !filepath.IsAbs(exe) {
|
if exe != "" && !filepath.IsAbs(exe) {
|
||||||
exe = filepath.Join(agentpaths.InstallRoot(), exe)
|
exe = filepath.Join(agentpaths.InstallRoot(), exe)
|
||||||
}
|
}
|
||||||
|
a.mu.Lock()
|
||||||
|
a.engineExe = exe
|
||||||
|
a.mu.Unlock()
|
||||||
logFile, _ := agentengine.LogFile(agentpaths.EngineLog())
|
logFile, _ := agentengine.LogFile(agentpaths.EngineLog())
|
||||||
a.sup = agentengine.New(agentengine.Options{
|
a.sup = agentengine.New(agentengine.Options{
|
||||||
Command: func(c context.Context) *exec.Cmd {
|
Command: func(c context.Context) *exec.Cmd {
|
||||||
@@ -146,13 +160,55 @@ func (a *App) startup(ctx context.Context) {
|
|||||||
// not run yet, starting the supervisor would loop on a missing executable
|
// not run yet, starting the supervisor would loop on a missing executable
|
||||||
// with nothing useful to say. The Start button still exists for the one
|
// with nothing useful to say. The Start button still exists for the one
|
||||||
// case where somebody has deliberately stopped it.
|
// case where somebody has deliberately stopped it.
|
||||||
if _, err := os.Stat(exe); err == nil {
|
if why := a.engineMissing(); why == "" {
|
||||||
a.sup.Start()
|
a.sup.Start()
|
||||||
} else {
|
} else {
|
||||||
log.Printf("engine not installed yet (%s); run behavision-setup, then Start", exe)
|
log.Printf("%s (looked for %s)", why, exe)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// engineMissing says, in a sentence somebody can act on, why recognition
|
||||||
|
// cannot start here - or "" when it can.
|
||||||
|
//
|
||||||
|
// It exists because the answer was only ever given at startup, to a log file
|
||||||
|
// nobody on a shop counter opens. Pressing Start went straight to the
|
||||||
|
// supervisor, which reported what exec reported:
|
||||||
|
//
|
||||||
|
// engine failed to start: fork/exec /private/var/folders/c2/.../
|
||||||
|
// AppTranslocation/500A5354-.../d/Behavision.app/Contents/MacOS/engine/
|
||||||
|
// behavision: no such file or directory
|
||||||
|
//
|
||||||
|
// Every word of that is true and none of it says "run the setup tool". One
|
||||||
|
// function, consulted by the startup path, the Start button and the status
|
||||||
|
// panel, so the three cannot give three different accounts of one fact.
|
||||||
|
func (a *App) engineMissing() string {
|
||||||
|
a.mu.RLock()
|
||||||
|
exe := a.engineExe
|
||||||
|
a.mu.RUnlock()
|
||||||
|
if exe == "" {
|
||||||
|
return "The recognition engine is not set up on this computer yet."
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(exe); err == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
msg := "The recognition engine is not installed on this computer yet. " +
|
||||||
|
"Run behavision-setup from the folder you unzipped, then press Start."
|
||||||
|
// macOS quarantines a downloaded app it cannot verify and runs it from a
|
||||||
|
// randomly named READ-ONLY copy - App Translocation. Every relative path
|
||||||
|
// then resolves inside that copy, which is why the engine folder appears
|
||||||
|
// to be missing from a bundle that plainly contains one, and why an
|
||||||
|
// install into it would not survive a restart. Detectable, unguessable,
|
||||||
|
// and fixed by one drag; saying nothing leaves somebody re-running a
|
||||||
|
// setup tool that cannot win.
|
||||||
|
if strings.Contains(exe, "/AppTranslocation/") {
|
||||||
|
msg = "macOS is running Behavision from a temporary read-only copy, " +
|
||||||
|
"because it was opened straight from Downloads. Move Behavision " +
|
||||||
|
"to your Applications folder and open it from there, then run " +
|
||||||
|
"behavision-setup."
|
||||||
|
}
|
||||||
|
return msg
|
||||||
|
}
|
||||||
|
|
||||||
// webhookURL is the loopback address the bridge is listening on, or empty
|
// webhookURL is the loopback address the bridge is listening on, or empty
|
||||||
// before it has started.
|
// before it has started.
|
||||||
func (a *App) webhookURL() string {
|
func (a *App) webhookURL() string {
|
||||||
@@ -234,7 +290,7 @@ func (a *App) startPipeline(ctx context.Context) {
|
|||||||
}
|
}
|
||||||
client, err := agentmqtt.NewClient(agentmqtt.ClientOptions{
|
client, err := agentmqtt.NewClient(agentmqtt.ClientOptions{
|
||||||
BrokerURL: a.cfg.BrokerURL,
|
BrokerURL: a.cfg.BrokerURL,
|
||||||
ClientID: "behavision-" + a.cfg.ClientID + "-" + a.cfg.SiteID,
|
ClientID: a.cfg.MQTTClientID(),
|
||||||
Username: a.cfg.BrokerUsername, Password: a.cfg.BrokerPassword,
|
Username: a.cfg.BrokerUsername, Password: a.cfg.BrokerPassword,
|
||||||
CAFile: a.cfg.BrokerCAFile, Log: logger,
|
CAFile: a.cfg.BrokerCAFile, Log: logger,
|
||||||
})
|
})
|
||||||
@@ -587,6 +643,11 @@ func (a *App) EngineStatus() EngineStatus {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
out.Error = err.Error()
|
out.Error = err.Error()
|
||||||
}
|
}
|
||||||
|
// The supervisor's own error is an exec failure; this replaces it with
|
||||||
|
// the reason, which is the part that tells somebody what to do.
|
||||||
|
if why := a.engineMissing(); why != "" {
|
||||||
|
out.Error = why
|
||||||
|
}
|
||||||
ctx, cancel := context.WithTimeout(a.ctx, 4*time.Second)
|
ctx, cancel := context.WithTimeout(a.ctx, 4*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
// A running process is not a working engine: on a memory-starved box the
|
// A running process is not a working engine: on a memory-starved box the
|
||||||
@@ -601,6 +662,13 @@ func (a *App) EngineStatus() EngineStatus {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (a *App) StartEngine() EngineStatus {
|
func (a *App) StartEngine() EngineStatus {
|
||||||
|
// Refused rather than attempted. Handing a missing path to the supervisor
|
||||||
|
// produces a retry loop and an exec error for a message.
|
||||||
|
if why := a.engineMissing(); why != "" {
|
||||||
|
st := a.EngineStatus()
|
||||||
|
st.Error = why
|
||||||
|
return st
|
||||||
|
}
|
||||||
if a.sup != nil {
|
if a.sup != nil {
|
||||||
a.sup.Start()
|
a.sup.Start()
|
||||||
}
|
}
|
||||||
@@ -646,6 +714,7 @@ func (a *App) Cameras() ([]map[string]any, error) {
|
|||||||
"id": c.ID, "camera_id": c.CameraID, "label": c.Label,
|
"id": c.ID, "camera_id": c.CameraID, "label": c.Label,
|
||||||
"site": c.Site, "enabled": c.Enabled,
|
"site": c.Site, "enabled": c.Enabled,
|
||||||
"connected": c.Connected, "last_seen_at": c.LastSeenAt,
|
"connected": c.Connected, "last_seen_at": c.LastSeenAt,
|
||||||
|
"state": c.State, "state_note": c.StateNote,
|
||||||
"snapshot": c.Snapshot, "snapshot_at": c.SnapshotAt,
|
"snapshot": c.Snapshot, "snapshot_at": c.SnapshotAt,
|
||||||
// What the screen keys off to hide Edit, Test and Check: this
|
// What the screen keys off to hide Edit, Test and Check: this
|
||||||
// camera is on a network this PC cannot reach.
|
// camera is on a network this PC cannot reach.
|
||||||
@@ -719,6 +788,23 @@ func (a *App) StreamURL(cameraID string) string {
|
|||||||
return fmt.Sprintf("http://%s/api/cameras/%s/stream.mjpeg", base, cameraID)
|
return fmt.Sprintf("http://%s/api/cameras/%s/stream.mjpeg", base, cameraID)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RemoteStreamURL is the live view of a camera in another building.
|
||||||
|
//
|
||||||
|
// The picture comes from head office's relay - the shop PC pushes frames
|
||||||
|
// outbound because nothing can reach in - and this app re-emits them as MJPEG
|
||||||
|
// on its own loopback, so a tile is an ordinary <img> either way. A screen
|
||||||
|
// therefore never has to know which building it is looking at.
|
||||||
|
//
|
||||||
|
// Empty when the relay is not running, and the caller shows the last snapshot
|
||||||
|
// instead. There is no useful fallback URL: the head-office endpoint needs
|
||||||
|
// this session's bearer, which an <img> cannot send.
|
||||||
|
func (a *App) RemoteStreamURL(cameraID string) string {
|
||||||
|
if !a.cloud.LoggedIn() {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return a.proxy.urlFor(cameraID, "live.mjpeg")
|
||||||
|
}
|
||||||
|
|
||||||
// ------------------------------------------------------------------- live --
|
// ------------------------------------------------------------------- live --
|
||||||
|
|
||||||
type LiveSnapshot struct {
|
type LiveSnapshot struct {
|
||||||
|
|||||||
40
desktop/frontend/dist/assets/index-6LYbNlbD.js
vendored
Normal file
40
desktop/frontend/dist/assets/index-6LYbNlbD.js
vendored
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
40
desktop/frontend/dist/assets/index-vAtlvw9l.js
vendored
40
desktop/frontend/dist/assets/index-vAtlvw9l.js
vendored
File diff suppressed because one or more lines are too long
4
desktop/frontend/dist/index.html
vendored
4
desktop/frontend/dist/index.html
vendored
@@ -4,8 +4,8 @@
|
|||||||
<meta charset="UTF-8" />
|
<meta charset="UTF-8" />
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
<title>Behavision</title>
|
<title>Behavision</title>
|
||||||
<script type="module" crossorigin src="./assets/index-vAtlvw9l.js"></script>
|
<script type="module" crossorigin src="./assets/index-6LYbNlbD.js"></script>
|
||||||
<link rel="stylesheet" crossorigin href="./assets/index-xWw5ie4A.css">
|
<link rel="stylesheet" crossorigin href="./assets/index-DOJ2bRrM.css">
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
<div id="root"></div>
|
<div id="root"></div>
|
||||||
|
|||||||
@@ -38,6 +38,9 @@ export const api = {
|
|||||||
startPlacement: (id, seconds) => call('StartPlacementCheck', id, seconds),
|
startPlacement: (id, seconds) => call('StartPlacementCheck', id, seconds),
|
||||||
placementResult: (id) => call('PlacementResult', id),
|
placementResult: (id) => call('PlacementResult', id),
|
||||||
streamURL: (id) => call('StreamURL', id),
|
streamURL: (id) => call('StreamURL', id),
|
||||||
|
// The live view of a camera in another building, relayed through head
|
||||||
|
// office. Empty when nobody is signed in.
|
||||||
|
remoteStreamURL: (id) => call('RemoteStreamURL', id),
|
||||||
|
|
||||||
live: () => call('Live'),
|
live: () => call('Live'),
|
||||||
pipelineStatus: () => call('PipelineStatus'),
|
pipelineStatus: () => call('PipelineStatus'),
|
||||||
|
|||||||
@@ -648,3 +648,18 @@ tr.click { cursor: pointer; } tr.click:hover td { background: var(--s2); }
|
|||||||
}
|
}
|
||||||
.viewing b { color: var(--ink); font-weight: 600; }
|
.viewing b { color: var(--ink); font-weight: 600; }
|
||||||
.viewing svg { flex: none; margin-top: 2px; color: var(--accent); }
|
.viewing svg { flex: none; margin-top: 2px; color: var(--accent); }
|
||||||
|
|
||||||
|
/* Watch live sits over the picture, opposite the connection pill. It is on
|
||||||
|
the tile rather than in the button row because it is about the picture, and
|
||||||
|
because the row it would otherwise join is hidden on a remote camera. */
|
||||||
|
.camview .btn.watch {
|
||||||
|
position: absolute;
|
||||||
|
right: 10px;
|
||||||
|
bottom: 10px;
|
||||||
|
background: rgba(0, 0, 0, .55);
|
||||||
|
border-color: rgba(255, 255, 255, .25);
|
||||||
|
color: #fff;
|
||||||
|
backdrop-filter: blur(6px);
|
||||||
|
}
|
||||||
|
.camview .btn.watch:hover { background: rgba(0, 0, 0, .72); }
|
||||||
|
.camview .btn.watch.on { background: var(--accent); border-color: var(--accent); color: #fff; }
|
||||||
|
|||||||
@@ -27,6 +27,19 @@ export default function Cameras() {
|
|||||||
// than one that is absent.
|
// than one that is absent.
|
||||||
const remote = cams.some(c => c.remote)
|
const remote = cams.some(c => c.remote)
|
||||||
const streams = useStreamURLs(remote ? [] : cams)
|
const streams = useStreamURLs(remote ? [] : cams)
|
||||||
|
// ONE camera at a time, and that is a cost decision rather than a layout
|
||||||
|
// one. A remote view makes the shop computer upload frames for as long as
|
||||||
|
// somebody is watching, so a grid that all went live at once would put an
|
||||||
|
// estate's worth of cameras on the wire because somebody opened a page.
|
||||||
|
const [watching, setWatching] = useState(null)
|
||||||
|
const [watchURL, setWatchURL] = useState('')
|
||||||
|
useEffect(() => {
|
||||||
|
let alive = true
|
||||||
|
if (!watching) { setWatchURL(''); return }
|
||||||
|
api.remoteStreamURL(watching).then(u => { if (alive) setWatchURL(u || '') })
|
||||||
|
.catch(() => { if (alive) setWatchURL('') })
|
||||||
|
return () => { alive = false }
|
||||||
|
}, [watching])
|
||||||
|
|
||||||
async function remove(cam) {
|
async function remove(cam) {
|
||||||
if (!confirm(`Remove ${cam.id}? Recognition from it stops immediately.`)) return
|
if (!confirm(`Remove ${cam.id}? Recognition from it stops immediately.`)) return
|
||||||
@@ -49,8 +62,9 @@ export default function Cameras() {
|
|||||||
|
|
||||||
{remote && <div className="viewing">
|
{remote && <div className="viewing">
|
||||||
<b>Viewing your shops from here.</b> These cameras are wired to the shop
|
<b>Viewing your shops from here.</b> These cameras are wired to the shop
|
||||||
computers, so they are set up and checked there. The picture is each
|
computers, so they are set up and checked there. Each tile shows that
|
||||||
camera's most recent frame, not live video.
|
camera's most recent frame; <b>Watch live</b> asks the shop computer to
|
||||||
|
send video for as long as you are looking.
|
||||||
</div>}
|
</div>}
|
||||||
|
|
||||||
{error && <div className="err"><Icon.Warning size={15} />{error}</div>}
|
{error && <div className="err"><Icon.Warning size={15} />{error}</div>}
|
||||||
@@ -66,7 +80,10 @@ export default function Cameras() {
|
|||||||
</div>
|
</div>
|
||||||
: <div className="camgrid">
|
: <div className="camgrid">
|
||||||
{cams.map(c => (
|
{cams.map(c => (
|
||||||
<CameraCard key={c.id} cam={c} stream={streams[c.id]}
|
<CameraCard key={c.id} cam={c}
|
||||||
|
stream={watching === c.id ? watchURL : streams[c.id]}
|
||||||
|
watching={watching === c.id}
|
||||||
|
onWatch={() => setWatching(watching === c.id ? null : c.id)}
|
||||||
onEdit={() => setEditing(c)} onCheck={() => setCheck(c.id)} onRemove={() => remove(c)} />
|
onEdit={() => setEditing(c)} onCheck={() => setCheck(c.id)} onRemove={() => remove(c)} />
|
||||||
))}
|
))}
|
||||||
</div>}
|
</div>}
|
||||||
@@ -78,14 +95,23 @@ export default function Cameras() {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
function CameraCard({ cam, stream, onEdit, onCheck, onRemove }) {
|
function CameraCard({ cam, stream, watching, onWatch, onEdit, onCheck, onRemove }) {
|
||||||
// Three states, not two, and the third is why `connected` is a pointer on
|
// Three states, not two, and the third is why `connected` is a pointer on
|
||||||
// the wire: null means no shop computer has reported on this camera yet,
|
// the wire: null means no shop computer has reported on this camera yet,
|
||||||
// which reads as waiting rather than as a fault to go and investigate.
|
// which reads as waiting rather than as a fault to go and investigate.
|
||||||
const conn = cam.connected === undefined || cam.connected === null
|
const conn = cam.remote
|
||||||
? (cam.remote ? { tone: 'idle', label: 'Waiting for the shop computer' }
|
// Four states, decided once by the server. `stale` is the one that was
|
||||||
: { tone: 'idle', label: 'Engine stopped' })
|
// missing: the shop computer reports nothing when it cannot reach its own
|
||||||
: cam.connected ? { tone: 'ok', label: 'Connected' } : { tone: 'bad', label: 'Not connecting' }
|
// engine, so its last report used to sit there reading Connected -
|
||||||
|
// measured at 34 minutes on the live estate.
|
||||||
|
? ({ connected: { tone: 'ok', label: 'Connected' },
|
||||||
|
not_connecting: { tone: 'bad', label: 'Not connecting' },
|
||||||
|
stale: { tone: 'warn', label: 'Not reporting' } }[cam.state]
|
||||||
|
|| { tone: 'idle', label: 'Waiting for the shop computer' })
|
||||||
|
: cam.connected === undefined || cam.connected === null
|
||||||
|
? { tone: 'idle', label: 'Engine stopped' }
|
||||||
|
: cam.connected ? { tone: 'ok', label: 'Connected' }
|
||||||
|
: { tone: 'bad', label: 'Not connecting' }
|
||||||
// The last placement verdict, so "proven" survives closing the sheet. Only
|
// The last placement verdict, so "proven" survives closing the sheet. Only
|
||||||
// `good` is a pass: marginal means half the visitors are silently discarded.
|
// `good` is a pass: marginal means half the visitors are silently discarded.
|
||||||
// Never asked for a remote camera: that answer lives on the shop computer,
|
// Never asked for a remote camera: that answer lives on the shop computer,
|
||||||
@@ -105,6 +131,9 @@ function CameraCard({ cam, stream, onEdit, onCheck, onRemove }) {
|
|||||||
? <img src={stream || shot} alt={cam.id} />
|
? <img src={stream || shot} alt={cam.id} />
|
||||||
: <div className="placeholder"><Icon.NoCamera size={34} /></div>}
|
: <div className="placeholder"><Icon.NoCamera size={34} /></div>}
|
||||||
<span className={`pill ${conn.tone === 'idle' ? '' : conn.tone} over`}><i className={`dot ${conn.tone}`} />{conn.label}</span>
|
<span className={`pill ${conn.tone === 'idle' ? '' : conn.tone} over`}><i className={`dot ${conn.tone}`} />{conn.label}</span>
|
||||||
|
{cam.remote && <button className={`btn sm watch ${watching ? 'on' : ''}`} onClick={onWatch}>
|
||||||
|
<Icon.Play size={13} />{watching ? 'Stop watching' : 'Watch live'}
|
||||||
|
</button>}
|
||||||
</div>
|
</div>
|
||||||
<div className="cambody">
|
<div className="cambody">
|
||||||
<div className="camtitle">
|
<div className="camtitle">
|
||||||
@@ -121,9 +150,13 @@ function CameraCard({ cam, stream, onEdit, onCheck, onRemove }) {
|
|||||||
</div>
|
</div>
|
||||||
{cam.remote
|
{cam.remote
|
||||||
? <div className="camproof">
|
? <div className="camproof">
|
||||||
<span className="note">{cam.snapshot?.available
|
<span className="note">{cam.state_note
|
||||||
? 'Last picture from the shop computer.'
|
? cam.state_note
|
||||||
: cam.snapshot?.reason || 'No picture yet from the shop computer.'}</span>
|
: watching
|
||||||
|
? 'Live from the shop computer. It uploads only while you watch.'
|
||||||
|
: cam.snapshot?.available
|
||||||
|
? 'Last picture from the shop computer. Watch live to see it now.'
|
||||||
|
: cam.snapshot?.reason || 'No picture yet from the shop computer.'}</span>
|
||||||
</div>
|
</div>
|
||||||
: <div className="camproof">
|
: <div className="camproof">
|
||||||
<span className={`tag ${proof.tone}`}>{proof.label}</span>
|
<span className={`tag ${proof.tone}`}>{proof.label}</span>
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/url"
|
"net/url"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -682,6 +683,12 @@ type RemoteCamera struct {
|
|||||||
Enabled bool `json:"enabled"`
|
Enabled bool `json:"enabled"`
|
||||||
Connected *bool `json:"connected"`
|
Connected *bool `json:"connected"`
|
||||||
LastSeenAt string `json:"last_seen_at"`
|
LastSeenAt string `json:"last_seen_at"`
|
||||||
|
// State is the server's single answer - connected / not_connecting /
|
||||||
|
// waiting / stale - and the screen renders that rather than deciding
|
||||||
|
// again from Connected. Two places deciding one fact is how a shop came
|
||||||
|
// out labelled Working, in green, above "2 of 3 cameras not connecting".
|
||||||
|
State string `json:"state"`
|
||||||
|
StateNote string `json:"state_note"`
|
||||||
Snapshot Photo `json:"snapshot"`
|
Snapshot Photo `json:"snapshot"`
|
||||||
SnapshotAt string `json:"snapshot_at"`
|
SnapshotAt string `json:"snapshot_at"`
|
||||||
}
|
}
|
||||||
@@ -746,3 +753,72 @@ func (c *Client) resolveShot(ctx context.Context, camID, at string, p Photo) Pho
|
|||||||
p.URL, p.Auth = uri, false
|
p.URL, p.Auth = uri, false
|
||||||
return p
|
return p
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CameraLive opens head office's live relay for one camera and returns the
|
||||||
|
// live SSE response for the caller to read and close.
|
||||||
|
//
|
||||||
|
// A response rather than frames, because the consumer is the app's own
|
||||||
|
// loopback relay: it re-emits these frames as MJPEG so an <img> can show them,
|
||||||
|
// and buffering the stream through a channel here would only add a place for
|
||||||
|
// frames to queue. A stale frame is worthless - the only one worth having is
|
||||||
|
// the newest - which is the whole reason LiveHub drops rather than queues.
|
||||||
|
//
|
||||||
|
// There is no client timeout on this request. A live view is endless by
|
||||||
|
// design and any deadline would cut the picture off mid-shift; the context is
|
||||||
|
// what ends it, when the viewer navigates away.
|
||||||
|
func (c *Client) CameraLive(ctx context.Context, cameraID string) (*http.Response, error) {
|
||||||
|
resp, err := c.liveOnce(ctx, cameraID)
|
||||||
|
if errors.Is(err, errTokenExpired) {
|
||||||
|
if rerr := c.Refresh(ctx); rerr != nil {
|
||||||
|
return nil, rerr
|
||||||
|
}
|
||||||
|
resp, err = c.liveOnce(ctx, cameraID)
|
||||||
|
}
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) liveOnce(ctx context.Context, cameraID string) (*http.Response, error) {
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
|
||||||
|
c.Base+"/api/cameras/"+url.PathEscape(cameraID)+"/live", nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
req.Header.Set("Accept", "text/event-stream")
|
||||||
|
c.mu.RLock()
|
||||||
|
tok := c.token
|
||||||
|
c.mu.RUnlock()
|
||||||
|
if tok == "" {
|
||||||
|
return nil, ErrUnauthorized
|
||||||
|
}
|
||||||
|
req.Header.Set("Authorization", "Bearer "+tok)
|
||||||
|
|
||||||
|
// c.http has a 30 s timeout, which covers the whole response and would
|
||||||
|
// therefore sever a working live view every thirty seconds - the same
|
||||||
|
// trap that made the server set WriteTimeout to zero for its own SSE
|
||||||
|
// endpoint. A dedicated client, with the dial bounded instead.
|
||||||
|
hc := &http.Client{Transport: &http.Transport{
|
||||||
|
DialContext: (&net.Dialer{Timeout: 10 * time.Second}).DialContext,
|
||||||
|
TLSHandshakeTimeout: 10 * time.Second,
|
||||||
|
}}
|
||||||
|
resp, err := hc.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cannot reach %s: %w", c.Base, err)
|
||||||
|
}
|
||||||
|
if resp.StatusCode == http.StatusUnauthorized {
|
||||||
|
var e struct {
|
||||||
|
Error string `json:"error"`
|
||||||
|
}
|
||||||
|
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8192))
|
||||||
|
resp.Body.Close()
|
||||||
|
_ = json.Unmarshal(body, &e)
|
||||||
|
if e.Error == "token_expired" {
|
||||||
|
return nil, errTokenExpired
|
||||||
|
}
|
||||||
|
return nil, ErrUnauthorized
|
||||||
|
}
|
||||||
|
if resp.StatusCode >= 400 {
|
||||||
|
resp.Body.Close()
|
||||||
|
return nil, fmt.Errorf("live view: %s", resp.Status)
|
||||||
|
}
|
||||||
|
return resp, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ package main
|
|||||||
// session and the bytes are fetched and handed over as an object URL.
|
// session and the bytes are fetched and handed over as an object URL.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"crypto/rand"
|
"crypto/rand"
|
||||||
"crypto/subtle"
|
"crypto/subtle"
|
||||||
"encoding/hex"
|
"encoding/hex"
|
||||||
@@ -64,6 +65,12 @@ type streamProxy struct {
|
|||||||
target string // engine origin, e.g. http://127.0.0.1:8010
|
target string // engine origin, e.g. http://127.0.0.1:8010
|
||||||
user string
|
user string
|
||||||
pass string
|
pass string
|
||||||
|
|
||||||
|
// Opens head office's live relay for one camera. Set on a computer that
|
||||||
|
// is signed in, whether or not an engine runs here - which is the whole
|
||||||
|
// point: watching a camera in another building is precisely the case
|
||||||
|
// where there is no engine on this machine to ask.
|
||||||
|
live func(ctx context.Context, cameraID string) (*http.Response, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
func newStreamProxy() *streamProxy { return &streamProxy{} }
|
func newStreamProxy() *streamProxy { return &streamProxy{} }
|
||||||
@@ -71,18 +78,49 @@ func newStreamProxy() *streamProxy { return &streamProxy{} }
|
|||||||
// start binds a loopback listener and begins relaying. Calling it again while
|
// start binds a loopback listener and begins relaying. Calling it again while
|
||||||
// running is a no-op, so a restarted engine cannot leave two listeners behind.
|
// running is a no-op, so a restarted engine cannot leave two listeners behind.
|
||||||
func (p *streamProxy) start(base, user, pass string) error {
|
func (p *streamProxy) start(base, user, pass string) error {
|
||||||
p.mu.Lock()
|
|
||||||
defer p.mu.Unlock()
|
|
||||||
if p.srv != nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
if !strings.HasPrefix(base, "http://") && !strings.HasPrefix(base, "https://") {
|
if !strings.HasPrefix(base, "http://") && !strings.HasPrefix(base, "https://") {
|
||||||
base = "http://" + base
|
base = "http://" + base
|
||||||
}
|
}
|
||||||
if _, err := url.Parse(base); err != nil {
|
if _, err := url.Parse(base); err != nil {
|
||||||
return fmt.Errorf("engine base %q: %w", base, err)
|
return fmt.Errorf("engine base %q: %w", base, err)
|
||||||
}
|
}
|
||||||
|
if err := p.bind(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
p.target = strings.TrimRight(base, "/")
|
||||||
|
p.user, p.pass = user, pass
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// watchRemote makes the relay able to serve head office's live view, and
|
||||||
|
// binds it if nothing else has.
|
||||||
|
//
|
||||||
|
// Separate from start() because the two are independent: a shop PC has both
|
||||||
|
// an engine and a session, an owner's laptop has only a session, and a PC
|
||||||
|
// still being set up has only an engine. Folding them together would mean a
|
||||||
|
// computer with no engine could not watch a camera at all - which is the one
|
||||||
|
// computer most likely to be trying to.
|
||||||
|
func (p *streamProxy) watchRemote(fn func(context.Context, string) (*http.Response, error)) error {
|
||||||
|
if err := p.bind(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
p.live = fn
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// bind starts the loopback listener once. Calling it again while running is a
|
||||||
|
// no-op, so neither a restarted engine nor a second sign-in can leave two
|
||||||
|
// listeners behind.
|
||||||
|
func (p *streamProxy) bind() error {
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
if p.srv != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// The engine's own credential exists precisely so that the live face feed
|
// The engine's own credential exists precisely so that the live face feed
|
||||||
// is never served open - CLAUDE.md is explicit that an unauthenticated
|
// is never served open - CLAUDE.md is explicit that an unauthenticated
|
||||||
@@ -105,8 +143,6 @@ func (p *streamProxy) start(base, user, pass string) error {
|
|||||||
|
|
||||||
p.ln = ln
|
p.ln = ln
|
||||||
p.token = hex.EncodeToString(raw)
|
p.token = hex.EncodeToString(raw)
|
||||||
p.target = strings.TrimRight(base, "/")
|
|
||||||
p.user, p.pass = user, pass
|
|
||||||
// No client timeout: an MJPEG stream is endless by design and any deadline
|
// No client timeout: an MJPEG stream is endless by design and any deadline
|
||||||
// would cut the picture off mid-shift. The request context ends it when
|
// would cut the picture off mid-shift. The request context ends it when
|
||||||
// the webview navigates away or the tile is replaced.
|
// the webview navigates away or the tile is replaced.
|
||||||
@@ -130,7 +166,7 @@ func (p *streamProxy) start(base, user, pass string) error {
|
|||||||
func (p *streamProxy) stop() {
|
func (p *streamProxy) stop() {
|
||||||
p.mu.Lock()
|
p.mu.Lock()
|
||||||
srv, ln := p.srv, p.ln
|
srv, ln := p.srv, p.ln
|
||||||
p.srv, p.ln, p.token = nil, nil, ""
|
p.srv, p.ln, p.token, p.live = nil, nil, "", nil
|
||||||
p.mu.Unlock()
|
p.mu.Unlock()
|
||||||
if srv != nil {
|
if srv != nil {
|
||||||
_ = srv.Close()
|
_ = srv.Close()
|
||||||
@@ -155,6 +191,7 @@ func (p *streamProxy) urlFor(cameraID, file string) string {
|
|||||||
func (p *streamProxy) handle(w http.ResponseWriter, r *http.Request) {
|
func (p *streamProxy) handle(w http.ResponseWriter, r *http.Request) {
|
||||||
p.mu.RLock()
|
p.mu.RLock()
|
||||||
token, target, user, pass, client := p.token, p.target, p.user, p.pass, p.client
|
token, target, user, pass, client := p.token, p.target, p.user, p.pass, p.client
|
||||||
|
liveFn := p.live
|
||||||
p.mu.RUnlock()
|
p.mu.RUnlock()
|
||||||
if token == "" || client == nil {
|
if token == "" || client == nil {
|
||||||
http.NotFound(w, r)
|
http.NotFound(w, r)
|
||||||
@@ -190,6 +227,17 @@ func (p *streamProxy) handle(w http.ResponseWriter, r *http.Request) {
|
|||||||
// calls; it is here so that adding a still later is a change to a screen
|
// calls; it is here so that adding a still later is a change to a screen
|
||||||
// rather than a change to the one file where a mistake is a credentialed
|
// rather than a change to the one file where a mistake is a credentialed
|
||||||
// proxy onto the biometric API.
|
// proxy onto the biometric API.
|
||||||
|
// Head office's relay, not the engine. The two are different machines and
|
||||||
|
// different credentials, so this returns rather than falling through.
|
||||||
|
if parts[3] == "live.mjpeg" {
|
||||||
|
if liveFn == nil {
|
||||||
|
http.Error(w, "not signed in to head office", http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
p.relayRemote(w, r, cameraID, liveFn)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
var enginePath string
|
var enginePath string
|
||||||
switch parts[3] {
|
switch parts[3] {
|
||||||
case "stream.mjpeg":
|
case "stream.mjpeg":
|
||||||
|
|||||||
149
desktop/stream_remote.go
Normal file
149
desktop/stream_remote.go
Normal file
@@ -0,0 +1,149 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// Watching a camera in another building, from the app.
|
||||||
|
//
|
||||||
|
// The shop PC sits behind a router with no inbound route, so nothing here can
|
||||||
|
// pull its MJPEG stream - that stream is served on the shop PC's own loopback
|
||||||
|
// and always will be. Head office's LiveHub is the way round it: the agent
|
||||||
|
// asks outbound whether anyone is watching and pushes JPEG frames up for
|
||||||
|
// exactly as long as somebody is. The head-office web app already consumes
|
||||||
|
// that; this is the same feed, for the app.
|
||||||
|
//
|
||||||
|
// It arrives as base64 frames over SSE, which an <img> cannot render, so this
|
||||||
|
// re-emits them as multipart MJPEG - which an <img> renders natively, through
|
||||||
|
// the relay that already exists for the local engine. That is what keeps ONE
|
||||||
|
// code path in the screens: a tile points at a loopback URL and does not know
|
||||||
|
// or care which building the picture came from.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"context"
|
||||||
|
"encoding/base64"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The boundary is ours to choose; it only has to be a string the JPEG bytes
|
||||||
|
// cannot contain, and a marker line never appears inside JPEG data.
|
||||||
|
const mjpegBoundary = "behavisionframe"
|
||||||
|
|
||||||
|
// A frame is base64, so ~1.33 bytes on the wire per byte of picture. The
|
||||||
|
// engine re-encodes to 640 px for the relay and those measure ~20 KB, so this
|
||||||
|
// is roughly a hundredfold headroom - large enough never to clip a real frame
|
||||||
|
// and small enough that a broken or hostile stream cannot grow this process's
|
||||||
|
// memory without bound.
|
||||||
|
const maxFrameLine = 8 << 20
|
||||||
|
|
||||||
|
func (p *streamProxy) relayRemote(w http.ResponseWriter, r *http.Request,
|
||||||
|
cameraID string, open func(context.Context, string) (*http.Response, error)) {
|
||||||
|
|
||||||
|
w.Header().Set("Content-Type", "multipart/x-mixed-replace; boundary="+mjpegBoundary)
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
flusher, _ := w.(http.Flusher)
|
||||||
|
|
||||||
|
// Send the headers NOW, before any frame exists. Go writes them on the
|
||||||
|
// first body write, so without this the whole response - status line
|
||||||
|
// included - waits for the shop computer to start pushing, and a viewer
|
||||||
|
// whose camera is slow to answer sees the REQUEST time out rather than a
|
||||||
|
// stream that has not painted yet. Measured against production: 30
|
||||||
|
// seconds and not even a Content-Type.
|
||||||
|
if flusher != nil {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reconnecting is normal, not an error. The server caps one push at five
|
||||||
|
// minutes so that a tab left open for a week cannot leave a shop
|
||||||
|
// uploading for a week - so a viewer who IS still there simply asks
|
||||||
|
// again. Doing it here rather than in the page is what lets the <img>
|
||||||
|
// survive the cap: it never sees the stream end.
|
||||||
|
sent := 0
|
||||||
|
for {
|
||||||
|
if r.Context().Err() != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
n, err := p.pumpRemote(w, flusher, r.Context(), cameraID, open)
|
||||||
|
sent += n
|
||||||
|
if r.Context().Err() != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Nothing was written and the attempt failed. Writing an error body
|
||||||
|
// now would be writing it into a multipart stream the <img> is
|
||||||
|
// already parsing, so the picture simply stays on whatever it last
|
||||||
|
// showed and the screen's own "not connecting" state is the report.
|
||||||
|
if err != nil && sent == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-r.Context().Done():
|
||||||
|
return
|
||||||
|
case <-time.After(1500 * time.Millisecond):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// pumpRemote runs one SSE connection to exhaustion and returns how many
|
||||||
|
// frames it forwarded.
|
||||||
|
func (p *streamProxy) pumpRemote(w http.ResponseWriter, flusher http.Flusher,
|
||||||
|
ctx context.Context, cameraID string,
|
||||||
|
open func(context.Context, string) (*http.Response, error)) (int, error) {
|
||||||
|
|
||||||
|
resp, err := open(ctx, cameraID)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
sc := bufio.NewScanner(resp.Body)
|
||||||
|
sc.Buffer(make([]byte, 0, 64*1024), maxFrameLine)
|
||||||
|
|
||||||
|
var event, data string
|
||||||
|
frames := 0
|
||||||
|
for sc.Scan() {
|
||||||
|
line := sc.Text()
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(line, "event: "):
|
||||||
|
event = strings.TrimSpace(line[7:])
|
||||||
|
case strings.HasPrefix(line, "data: "):
|
||||||
|
data = line[6:]
|
||||||
|
case line == "":
|
||||||
|
// End of one SSE event. `waiting` means head office has us
|
||||||
|
// registered and the shop PC has not started pushing yet - a real
|
||||||
|
// second or two while the agent is asked, and nothing to draw.
|
||||||
|
if event == "frame" && data != "" {
|
||||||
|
if err := writeMJPEGFrame(w, flusher, data); err != nil {
|
||||||
|
return frames, err // the webview went away
|
||||||
|
}
|
||||||
|
frames++
|
||||||
|
}
|
||||||
|
event, data = "", ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return frames, sc.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeMJPEGFrame(w http.ResponseWriter, flusher http.Flusher, b64 string) error {
|
||||||
|
jpg, err := base64.StdEncoding.DecodeString(b64)
|
||||||
|
if err != nil || len(jpg) == 0 {
|
||||||
|
// One malformed frame is not a reason to tear down a working view.
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := fmt.Fprintf(w,
|
||||||
|
"--%s\r\nContent-Type: image/jpeg\r\nContent-Length: %d\r\n\r\n",
|
||||||
|
mjpegBoundary, len(jpg)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err := w.Write(jpg); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err := w.Write([]byte("\r\n")); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Flushed per frame. Anything held waiting for a full buffer is a tile
|
||||||
|
// that stays blank, which is indistinguishable from the view not working.
|
||||||
|
if flusher != nil {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
103
desktop/stream_remote_live_test.go
Normal file
103
desktop/stream_remote_live_test.go
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/loyaly/behavision-desktop/internal/cloud"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The whole chain against the real head office and a real shop computer:
|
||||||
|
//
|
||||||
|
// TEST_CLOUD_EMAIL=... TEST_CLOUD_PASSWORD=... \
|
||||||
|
// go test ./desktop/ -run RemoteLive -v
|
||||||
|
//
|
||||||
|
// Everything in stream_remote_test.go proves the relay against a fake that
|
||||||
|
// agrees with me. Only this proves the part that cannot be faked: that a shop
|
||||||
|
// computer behind a router with no inbound route actually pushes frames when
|
||||||
|
// asked, that they survive base64 and SSE, and that what comes out of the
|
||||||
|
// loopback relay is a multipart stream an <img> will paint.
|
||||||
|
//
|
||||||
|
// It also costs something to run, which is why it is opt-in: watching makes
|
||||||
|
// the shop computer upload for as long as the test reads.
|
||||||
|
func TestRemoteLiveFromProduction(t *testing.T) {
|
||||||
|
email, pass := os.Getenv("TEST_CLOUD_EMAIL"), os.Getenv("TEST_CLOUD_PASSWORD")
|
||||||
|
if email == "" || pass == "" {
|
||||||
|
t.Skip("set TEST_CLOUD_EMAIL and TEST_CLOUD_PASSWORD to run against production")
|
||||||
|
}
|
||||||
|
base := os.Getenv("TEST_CLOUD_URL")
|
||||||
|
if base == "" {
|
||||||
|
base = "https://mcp.loyaly.ai"
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
c := cloud.New(base)
|
||||||
|
if _, err := c.Login(ctx, email, pass); err != nil {
|
||||||
|
t.Fatalf("login: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
cams, err := c.RemoteCameras(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("cameras: %v", err)
|
||||||
|
}
|
||||||
|
t.Logf("%d cameras", len(cams))
|
||||||
|
target := os.Getenv("TEST_CLOUD_CAMERA")
|
||||||
|
for _, cam := range cams {
|
||||||
|
conn := "waiting"
|
||||||
|
if cam.Connected != nil {
|
||||||
|
conn = map[bool]string{true: "connected", false: "not connecting"}[*cam.Connected]
|
||||||
|
}
|
||||||
|
t.Logf(" %-10s %-16s %-15s snapshot=%v", cam.CameraID, cam.Site, conn, cam.Snapshot.Available)
|
||||||
|
if target == "" && cam.Connected != nil && *cam.Connected {
|
||||||
|
target = cam.CameraID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if target == "" {
|
||||||
|
t.Skip("no connected camera to watch")
|
||||||
|
}
|
||||||
|
|
||||||
|
p := newStreamProxy()
|
||||||
|
if err := p.watchRemote(c.CameraLive); err != nil {
|
||||||
|
t.Fatalf("watchRemote: %v", err)
|
||||||
|
}
|
||||||
|
defer p.stop()
|
||||||
|
|
||||||
|
rctx, rcancel := context.WithTimeout(ctx, 30*time.Second)
|
||||||
|
defer rcancel()
|
||||||
|
req, _ := http.NewRequestWithContext(rctx, http.MethodGet, p.urlFor(target, "live.mjpeg"), nil)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET relay: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
start := time.Now()
|
||||||
|
acc, buf, frames := make([]byte, 0, 1<<20), make([]byte, 32*1024), 0
|
||||||
|
for frames < 10 {
|
||||||
|
n, rerr := resp.Body.Read(buf)
|
||||||
|
acc = append(acc, buf[:n]...)
|
||||||
|
frames = bytes.Count(acc, []byte("--"+mjpegBoundary))
|
||||||
|
if rerr != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
el := time.Since(start)
|
||||||
|
t.Logf("watching %q: %d frames, %d bytes, %.1fs (%.1f fps, %.0f KB/s)",
|
||||||
|
target, frames, len(acc), el.Seconds(),
|
||||||
|
float64(frames)/el.Seconds(), float64(len(acc))/el.Seconds()/1024)
|
||||||
|
|
||||||
|
if frames < 3 {
|
||||||
|
t.Fatalf("got %d frames from a connected camera - the shop computer is "+
|
||||||
|
"not answering head office's request to push", frames)
|
||||||
|
}
|
||||||
|
// Bytes that are actually a picture, not a framing header that happens to
|
||||||
|
// be well formed. A JPEG begins FFD8.
|
||||||
|
if !bytes.Contains(acc, []byte{0xFF, 0xD8, 0xFF}) {
|
||||||
|
t.Error("no JPEG start marker anywhere in the stream")
|
||||||
|
}
|
||||||
|
}
|
||||||
209
desktop/stream_remote_test.go
Normal file
209
desktop/stream_remote_test.go
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/base64"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// jpg is a byte sequence that is not valid JPEG and does not need to be: what
|
||||||
|
// is under test is that the bytes arrive intact and framed, not that a decoder
|
||||||
|
// likes them.
|
||||||
|
var jpg = []byte{0xFF, 0xD8, 'h', 'e', 'l', 'l', 'o', 0xFF, 0xD9}
|
||||||
|
|
||||||
|
// sseServer answers head office's live endpoint with `pushes` frames and then
|
||||||
|
// ends the response, which is what the server's five-minute cap does.
|
||||||
|
func sseServer(t *testing.T, frames int, hits *int32) *httptest.Server {
|
||||||
|
t.Helper()
|
||||||
|
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
atomic.AddInt32(hits, 1)
|
||||||
|
w.Header().Set("Content-Type", "text/event-stream")
|
||||||
|
fl, _ := w.(http.Flusher)
|
||||||
|
// Registered, nothing being pushed yet. Nothing may be drawn for it.
|
||||||
|
fmt.Fprint(w, "event: waiting\ndata: \n\n")
|
||||||
|
if fl != nil {
|
||||||
|
fl.Flush()
|
||||||
|
}
|
||||||
|
for i := 0; i < frames; i++ {
|
||||||
|
fmt.Fprintf(w, "event: frame\ndata: %s\n\n",
|
||||||
|
base64.StdEncoding.EncodeToString(jpg))
|
||||||
|
if fl != nil {
|
||||||
|
fl.Flush()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
func openerFor(srv *httptest.Server) func(context.Context, string) (*http.Response, error) {
|
||||||
|
return func(ctx context.Context, cam string) (*http.Response, error) {
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+"/live/"+cam, nil)
|
||||||
|
return http.DefaultClient.Do(req)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The whole point: base64 frames over SSE are not something an <img> can show,
|
||||||
|
// and a multipart MJPEG stream is. Without this the app could only ever show a
|
||||||
|
// still, on exactly the computers that cannot reach the camera any other way.
|
||||||
|
func TestRemoteFramesReachTheWebviewAsMJPEG(t *testing.T) {
|
||||||
|
var hits int32
|
||||||
|
srv := sseServer(t, 3, &hits)
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
p := newStreamProxy()
|
||||||
|
if err := p.watchRemote(openerFor(srv)); err != nil {
|
||||||
|
t.Fatalf("watchRemote: %v", err)
|
||||||
|
}
|
||||||
|
defer p.stop()
|
||||||
|
|
||||||
|
u := p.urlFor("cam2", "live.mjpeg")
|
||||||
|
if u == "" {
|
||||||
|
t.Fatal("no relay url; the proxy did not bind")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The relay reconnects for as long as the viewer is there, so the read is
|
||||||
|
// bounded by us rather than by the stream ending - exactly as an <img>
|
||||||
|
// would behave.
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET relay: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
if ct := resp.Header.Get("Content-Type"); !strings.HasPrefix(ct, "multipart/x-mixed-replace") {
|
||||||
|
t.Fatalf("Content-Type = %q, an <img> will not treat that as a stream", ct)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read the first three frames' worth and stop; the relay would otherwise
|
||||||
|
// go on reconnecting forever, which is the behaviour being relied on.
|
||||||
|
want := append([]byte(fmt.Sprintf("--%s\r\nContent-Type: image/jpeg\r\nContent-Length: %d\r\n\r\n",
|
||||||
|
mjpegBoundary, len(jpg))), jpg...)
|
||||||
|
got := make([]byte, 0, 4096)
|
||||||
|
buf := make([]byte, 512)
|
||||||
|
for len(got) < 3*len(want) {
|
||||||
|
n, rerr := resp.Body.Read(buf)
|
||||||
|
got = append(got, buf[:n]...)
|
||||||
|
if rerr != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if n := bytes.Count(got, []byte("--"+mjpegBoundary)); n < 3 {
|
||||||
|
t.Fatalf("got %d frames in %d bytes, want at least 3", n, len(got))
|
||||||
|
}
|
||||||
|
if !bytes.Contains(got, want) {
|
||||||
|
t.Errorf("a frame was not framed as expected:\n%q", got[:min(len(got), 300)])
|
||||||
|
}
|
||||||
|
// `waiting` is a real state - head office has us registered and the shop
|
||||||
|
// computer has not started pushing - and there is nothing to draw for it.
|
||||||
|
// Emitting an empty part would blank a tile that already had a picture.
|
||||||
|
if bytes.Contains(got, []byte("Content-Length: 0")) {
|
||||||
|
t.Error("an empty frame was written for a waiting event")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The server caps one push at five minutes so a tab left open for a week
|
||||||
|
// cannot leave a shop uploading for a week. Reconnecting is therefore a normal
|
||||||
|
// event, and doing it here rather than in the page is what lets the <img>
|
||||||
|
// survive the cap - it never sees the stream end.
|
||||||
|
func TestTheRelayReconnectsWhenHeadOfficeEndsAPush(t *testing.T) {
|
||||||
|
var hits int32
|
||||||
|
srv := sseServer(t, 1, &hits)
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
p := newStreamProxy()
|
||||||
|
if err := p.watchRemote(openerFor(srv)); err != nil {
|
||||||
|
t.Fatalf("watchRemote: %v", err)
|
||||||
|
}
|
||||||
|
defer p.stop()
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, p.urlFor("cam2", "live.mjpeg"), nil)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET relay: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
// Two frames means two pushes, because each push carries exactly one.
|
||||||
|
seen, buf := 0, make([]byte, 256)
|
||||||
|
acc := make([]byte, 0, 2048)
|
||||||
|
for seen < 2 {
|
||||||
|
n, rerr := resp.Body.Read(buf)
|
||||||
|
acc = append(acc, buf[:n]...)
|
||||||
|
seen = bytes.Count(acc, []byte("--"+mjpegBoundary))
|
||||||
|
if rerr != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if seen < 2 {
|
||||||
|
t.Fatalf("got %d frames across reconnects, want 2", seen)
|
||||||
|
}
|
||||||
|
if got := atomic.LoadInt32(&hits); got < 2 {
|
||||||
|
t.Errorf("head office was asked %d times, want at least 2", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Signed out, the relay must not pretend. There is no fallback URL to offer
|
||||||
|
// either: the head-office endpoint needs this session's bearer, which an <img>
|
||||||
|
// cannot send - so a tile that silently failed would be the only alternative.
|
||||||
|
func TestTheRelayRefusesWhenNobodyIsSignedIn(t *testing.T) {
|
||||||
|
p := newStreamProxy()
|
||||||
|
if err := p.watchRemote(nil); err != nil {
|
||||||
|
t.Fatalf("watchRemote: %v", err)
|
||||||
|
}
|
||||||
|
defer p.stop()
|
||||||
|
|
||||||
|
resp, err := http.Get(p.urlFor("cam2", "live.mjpeg"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET relay: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
io.Copy(io.Discard, resp.Body)
|
||||||
|
if resp.StatusCode != http.StatusBadGateway {
|
||||||
|
t.Errorf("status = %d, want 502", resp.StatusCode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The relay is credentialed - it is a path to a live view of a shop floor -
|
||||||
|
// and the token is the only thing standing between another local process and
|
||||||
|
// it. live.mjpeg must be behind exactly the same door as the engine routes.
|
||||||
|
func TestTheRemoteRouteIsBehindTheSameToken(t *testing.T) {
|
||||||
|
var hits int32
|
||||||
|
srv := sseServer(t, 1, &hits)
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
p := newStreamProxy()
|
||||||
|
if err := p.watchRemote(openerFor(srv)); err != nil {
|
||||||
|
t.Fatalf("watchRemote: %v", err)
|
||||||
|
}
|
||||||
|
defer p.stop()
|
||||||
|
|
||||||
|
// The right shape, the wrong value.
|
||||||
|
parts := strings.Split(p.urlFor("cam2", "live.mjpeg"), "/")
|
||||||
|
parts[4] = strings.Repeat("0", len(parts[4]))
|
||||||
|
bad := strings.Join(parts, "/")
|
||||||
|
|
||||||
|
resp, err := http.Get(bad)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET relay: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
io.Copy(io.Discard, resp.Body)
|
||||||
|
if resp.StatusCode != http.StatusNotFound {
|
||||||
|
t.Errorf("status = %d, want 404 - and 404 rather than 403, because there is nothing here to tell an unwelcome caller they found the right door", resp.StatusCode)
|
||||||
|
}
|
||||||
|
if atomic.LoadInt32(&hits) != 0 {
|
||||||
|
t.Error("a request with the wrong token still made the shop computer upload")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,11 +1,12 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "behavision"
|
name = "behavision"
|
||||||
version = "1.1.0"
|
dynamic = ["version"]
|
||||||
description = "Production face recognition over RTSP"
|
description = "Production face recognition over RTSP"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"numpy>=1.26,<2.0",
|
# See requirements.txt for why numpy is uncapped and opencv is not.
|
||||||
"opencv-python>=4.8.1",
|
"numpy>=1.26,<3.0",
|
||||||
|
"opencv-python>=4.8.1,<5",
|
||||||
"onnxruntime>=1.16",
|
"onnxruntime>=1.16",
|
||||||
"fastapi>=0.110",
|
"fastapi>=0.110",
|
||||||
"uvicorn>=0.29",
|
"uvicorn>=0.29",
|
||||||
@@ -31,3 +32,9 @@ behavision = ["static/*"]
|
|||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
testpaths = ["tests"]
|
testpaths = ["tests"]
|
||||||
|
|
||||||
|
# One version for the package and the wheel. Two literals in two files is how
|
||||||
|
# they came to disagree (1.0.0 here, 1.1.0 there) and how neither tracked a
|
||||||
|
# release.
|
||||||
|
[tool.setuptools.dynamic]
|
||||||
|
version = {attr = "behavision.__version__"}
|
||||||
|
|||||||
21
release.sh
21
release.sh
@@ -50,7 +50,28 @@ step "2. Agent and setup tool"
|
|||||||
step "3. Engine source and wheel"
|
step "3. Engine source and wheel"
|
||||||
# The wheel is built with the checkout's own interpreter; requires-python is a
|
# The wheel is built with the checkout's own interpreter; requires-python is a
|
||||||
# statement about the SHOP PC, which setup enforces when it finds Python there.
|
# statement about the SHOP PC, which setup enforces when it finds Python there.
|
||||||
|
# Stamp the tag into the wheel version. Without this every release built
|
||||||
|
# behavision-1.1.0-py3-none-any.whl, and `pip install --upgrade` on a machine
|
||||||
|
# that already had 1.1.0 is a no-op - so an engine fix reached nobody who had
|
||||||
|
# ever run setup, while the Go binaries beside it updated normally. Nothing
|
||||||
|
# reported a version either, so there was no way to tell a shop PC three
|
||||||
|
# releases behind from a current one.
|
||||||
|
#
|
||||||
|
# v0.5.6-demo -> 0.5.6+demo, which is valid PEP 440: a local segment takes
|
||||||
|
# alphanumerics and dots, never hyphens.
|
||||||
|
TMPVER=$(mktemp)
|
||||||
|
PEP440=$(printf '%s' "${TAG#v}" | sed 's/-/+/; s/[^0-9A-Za-z.+]/./g')
|
||||||
|
trap 'git checkout -- behavision/__init__.py 2>/dev/null || true; rm -f "$TMPVER"' EXIT
|
||||||
|
# A temp file rather than `sed -i`, whose argument handling differs between
|
||||||
|
# BSD and GNU - this script is run from a Mac today and that is not a reason
|
||||||
|
# to plant a portability trap in a release path.
|
||||||
|
sed "s/^__version__ = .*/__version__ = \"$PEP440\"/" behavision/__init__.py > "$TMPVER"
|
||||||
|
cat "$TMPVER" > behavision/__init__.py
|
||||||
.venv/bin/python -m pip wheel --no-deps --ignore-requires-python -q -w "$STAGE/engine-src" . 2>&1 | grep -v "DEPRECATION\|WARNING: Ignoring" || true
|
.venv/bin/python -m pip wheel --no-deps --ignore-requires-python -q -w "$STAGE/engine-src" . 2>&1 | grep -v "DEPRECATION\|WARNING: Ignoring" || true
|
||||||
|
git checkout -- behavision/__init__.py
|
||||||
|
rm -f "$TMPVER"
|
||||||
|
trap - EXIT
|
||||||
|
echo " engine wheel version $PEP440"
|
||||||
ls "$STAGE"/engine-src/behavision-*.whl >/dev/null || { echo "wheel was not built" >&2; exit 1; }
|
ls "$STAGE"/engine-src/behavision-*.whl >/dev/null || { echo "wheel was not built" >&2; exit 1; }
|
||||||
cp pyproject.toml requirements.txt "$STAGE/engine-src/"
|
cp pyproject.toml requirements.txt "$STAGE/engine-src/"
|
||||||
mkdir -p "$STAGE/engine-src/config" && cp config/default.yaml "$STAGE/engine-src/config/"
|
mkdir -p "$STAGE/engine-src/config" && cp config/default.yaml "$STAGE/engine-src/config/"
|
||||||
|
|||||||
@@ -1,5 +1,31 @@
|
|||||||
numpy>=1.26,<2.0
|
# numpy 2 is allowed, and that is what lets this install on a current Python.
|
||||||
opencv-python>=4.8.1
|
# `<2.0` capped the resolver at numpy 1.26.4, whose newest wheel is cp312, so
|
||||||
|
# on a Mac with Python 3.14 pip fell back to BUILDING numpy from source and
|
||||||
|
# died in clang - two screens of C compiler output on a shop counter, for a
|
||||||
|
# version choice made silently by behavision-setup.
|
||||||
|
#
|
||||||
|
# Measured before changing it, nine runs of the detector guard per combination
|
||||||
|
# on one machine:
|
||||||
|
#
|
||||||
|
# numpy 1.26 / cv2 4.11 9 passed, 0 crashed
|
||||||
|
# numpy 2.0 / cv2 4.11 8 passed, 1 crashed
|
||||||
|
# numpy 1.26 / cv2 4.14 3 passed, 6 crashed
|
||||||
|
# numpy 2.0 / cv2 4.14 2 passed, 7 crashed
|
||||||
|
#
|
||||||
|
# numpy is not the variable there; OpenCV is. The crash was a test racing a
|
||||||
|
# shared cv2.FaceDetectorYN on purpose - undefined behaviour in C++, which 4.11
|
||||||
|
# usually turned into an exception and 4.14 usually turns into a segfault. The
|
||||||
|
# product never shares one (Engine._build_worker builds a detector per camera),
|
||||||
|
# and the test now runs that race out of process. With that fixed, the whole
|
||||||
|
# suite is 226 passed / 2 skipped on numpy 2.0.2, five runs out of five.
|
||||||
|
#
|
||||||
|
# opencv IS capped, and the two are not the same call. Everything above was
|
||||||
|
# measured on 4.x; OpenCV 5.0 is a major release this project has never run a
|
||||||
|
# real camera or an emotion model through, and an uncapped `>=4.8.1` means
|
||||||
|
# every NEW install silently gets it while every existing one keeps 4.11. Lift
|
||||||
|
# it after running a camera on 5.x, not before.
|
||||||
|
numpy>=1.26,<3.0
|
||||||
|
opencv-python>=4.8.1,<5
|
||||||
onnxruntime>=1.16
|
onnxruntime>=1.16
|
||||||
fastapi>=0.110
|
fastapi>=0.110
|
||||||
uvicorn>=0.29
|
uvicorn>=0.29
|
||||||
|
|||||||
96
server/internal/api/camera_state_test.go
Normal file
96
server/internal/api/camera_state_test.go
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func ptr(b bool) *bool { return &b }
|
||||||
|
|
||||||
|
// The state this was written for. Two cameras read "Connected", in green, on
|
||||||
|
// the live estate thirty-four minutes after the shop computer had stopped
|
||||||
|
// being able to see either of them - because the agent correctly reports
|
||||||
|
// nothing when it cannot reach the engine, and the last value it sent stays
|
||||||
|
// in the database looking current.
|
||||||
|
func TestAStaleReportIsNotAConnectedCamera(t *testing.T) {
|
||||||
|
now := time.Date(2026, 9, 30, 11, 13, 0, 0, time.UTC)
|
||||||
|
c := Camera{Connected: ptr(true),
|
||||||
|
LastSeenAt: now.Add(-34 * time.Minute).Format(time.RFC3339)}
|
||||||
|
c.CameraState(now)
|
||||||
|
|
||||||
|
if c.State != CameraStale {
|
||||||
|
t.Errorf("state = %q, want %q", c.State, CameraStale)
|
||||||
|
}
|
||||||
|
// Cleared, not merely overruled. A stale true left in place stays
|
||||||
|
// available to every client that reads the field directly, and leaves two
|
||||||
|
// fields on one object disagreeing.
|
||||||
|
if c.Connected != nil {
|
||||||
|
t.Errorf("connected = %v, want null - nobody currently knows", *c.Connected)
|
||||||
|
}
|
||||||
|
if c.StateNote == "" {
|
||||||
|
t.Error("a stale camera said nothing about what to do")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// One missed report is a dropped packet. Warning on it would put an alarm on a
|
||||||
|
// healthy estate every few minutes, and an indicator that cries wolf is one
|
||||||
|
// people learn to ignore.
|
||||||
|
func TestOneMissedReportIsStillConnected(t *testing.T) {
|
||||||
|
now := time.Now()
|
||||||
|
c := Camera{Connected: ptr(true),
|
||||||
|
LastSeenAt: now.Add(-90 * time.Second).Format(time.RFC3339)}
|
||||||
|
c.CameraState(now)
|
||||||
|
if c.State != CameraConnected {
|
||||||
|
t.Errorf("state = %q after 90s, want %q", c.State, CameraConnected)
|
||||||
|
}
|
||||||
|
if c.Connected == nil || !*c.Connected {
|
||||||
|
t.Error("a fresh report lost its connected flag")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// "Nobody has ever told us" and "nobody has told us lately" need different
|
||||||
|
// sentences: the first is a camera head office added a minute ago and the
|
||||||
|
// shop computer has not picked up, the second is a shop computer that has
|
||||||
|
// stopped. Sending an installer to the wrong one wastes a journey.
|
||||||
|
func TestNeverReportedIsNotTheSameAsStopped(t *testing.T) {
|
||||||
|
now := time.Now()
|
||||||
|
var never Camera
|
||||||
|
never.CameraState(now)
|
||||||
|
if never.State != CameraWaiting {
|
||||||
|
t.Errorf("state = %q, want %q", never.State, CameraWaiting)
|
||||||
|
}
|
||||||
|
|
||||||
|
stopped := Camera{LastSeenAt: now.Add(-time.Hour).Format(time.RFC3339)}
|
||||||
|
stopped.CameraState(now)
|
||||||
|
if stopped.State == never.State {
|
||||||
|
t.Fatal("a camera nobody has reported and one that stopped read the same")
|
||||||
|
}
|
||||||
|
if stopped.StateNote == never.StateNote {
|
||||||
|
t.Error("two states that need different actions gave the same advice")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A camera the shop computer CAN see and cannot open is the one case where
|
||||||
|
// "check the cabling" is the right advice, and it must stay distinguishable
|
||||||
|
// from the three where it is not.
|
||||||
|
func TestAFreshFailureSaysCheckTheCamera(t *testing.T) {
|
||||||
|
now := time.Now()
|
||||||
|
c := Camera{Connected: ptr(false), LastSeenAt: now.Format(time.RFC3339)}
|
||||||
|
c.CameraState(now)
|
||||||
|
if c.State != CameraNotConnect {
|
||||||
|
t.Errorf("state = %q, want %q", c.State, CameraNotConnect)
|
||||||
|
}
|
||||||
|
if c.Connected == nil || *c.Connected {
|
||||||
|
t.Error("a reported failure must stay false, not become unknown")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An unparseable timestamp is not a working camera. It should not be possible,
|
||||||
|
// which is exactly why it must not fall through to "connected".
|
||||||
|
func TestAnUnreadableTimestampIsStale(t *testing.T) {
|
||||||
|
c := Camera{Connected: ptr(true), LastSeenAt: "not a time"}
|
||||||
|
c.CameraState(time.Now())
|
||||||
|
if c.State != CameraStale || c.Connected != nil {
|
||||||
|
t.Errorf("state = %q connected = %v, want stale and unknown", c.State, c.Connected)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -655,6 +655,11 @@ type Camera struct {
|
|||||||
Snapshot Image `json:"snapshot"`
|
Snapshot Image `json:"snapshot"`
|
||||||
SnapshotAt string `json:"snapshot_at,omitempty"`
|
SnapshotAt string `json:"snapshot_at,omitempty"`
|
||||||
|
|
||||||
|
// State is the ONE answer a screen should render, because there are four
|
||||||
|
// of them and only three were ever expressed. See CameraState.
|
||||||
|
State string `json:"state"`
|
||||||
|
StateNote string `json:"state_note,omitempty"`
|
||||||
|
|
||||||
// Check is the last attempt to prove this camera works. Always present so
|
// Check is the last attempt to prove this camera works. Always present so
|
||||||
// a client can tell "never checked" from "checked and failed" without
|
// a client can tell "never checked" from "checked and failed" without
|
||||||
// guessing from an absent field.
|
// guessing from an absent field.
|
||||||
@@ -962,3 +967,70 @@ type DeviceSession struct {
|
|||||||
// hand.
|
// hand.
|
||||||
Current bool `json:"current"`
|
Current bool `json:"current"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Camera states, and why a fourth one had to exist.
|
||||||
|
//
|
||||||
|
// A shop computer reports each camera's state about once a minute. When it
|
||||||
|
// cannot reach the recognition engine it reports NOTHING - correctly, because
|
||||||
|
// it has nothing to say - and the last value it sent stays in the database
|
||||||
|
// unchanged. Measured on the live estate: two cameras reading **Connected**,
|
||||||
|
// in green, thirty-four minutes after the shop computer had stopped being
|
||||||
|
// able to see either of them, while the heartbeat from the same PC said
|
||||||
|
// 0 of 0 cameras. Both surfaces were reading stored fields and disagreeing.
|
||||||
|
//
|
||||||
|
// `false` could not be the answer. It means "this camera is not connecting",
|
||||||
|
// which sends an installer to check cabling on a camera that was working
|
||||||
|
// perfectly the last time anybody could ask it. The honest statement is that
|
||||||
|
// nobody currently knows - and that is a different sentence from "nobody has
|
||||||
|
// ever told us", which is what an unreported camera needs. Two states that
|
||||||
|
// need different actions must never share a word; the same rule that keeps
|
||||||
|
// `artifact` apart from `no_faces` in the commissioning verdicts.
|
||||||
|
const (
|
||||||
|
CameraConnected = "connected" // reported recently, and working
|
||||||
|
CameraNotConnect = "not_connecting" // reported recently, and not
|
||||||
|
CameraWaiting = "waiting" // no shop computer has ever reported
|
||||||
|
CameraStale = "stale" // reported once, and not lately
|
||||||
|
)
|
||||||
|
|
||||||
|
// CameraStaleAfter is five missed reports, not one.
|
||||||
|
//
|
||||||
|
// The agent reports on a 60 s tick, so one miss is a dropped packet or a slow
|
||||||
|
// upload. Calling that stale would put a warning on a healthy estate every few
|
||||||
|
// minutes, and an indicator that cries wolf is one people learn to ignore -
|
||||||
|
// which is the same reasoning that makes a site offline after three missed
|
||||||
|
// heartbeats rather than one.
|
||||||
|
const CameraStaleAfter = 5 * time.Minute
|
||||||
|
|
||||||
|
// CameraState decides the four states, and nulls Connected when it is not
|
||||||
|
// entitled to an opinion.
|
||||||
|
//
|
||||||
|
// Connected is CLEARED rather than left alone on purpose. Leaving a stale true
|
||||||
|
// in place would keep the lie available to every client that reads the field
|
||||||
|
// directly - a mobile app, a script, an older build of our own desktop app -
|
||||||
|
// and would leave two fields on one object disagreeing, which is precisely how
|
||||||
|
// the shops screen once came out labelled Working in green above "2 of 3
|
||||||
|
// cameras not connecting".
|
||||||
|
func (c *Camera) CameraState(now time.Time) {
|
||||||
|
switch {
|
||||||
|
case c.LastSeenAt == "":
|
||||||
|
c.State = CameraWaiting
|
||||||
|
c.StateNote = "No shop computer has reported on this camera yet."
|
||||||
|
c.Connected = nil
|
||||||
|
return
|
||||||
|
}
|
||||||
|
seen, err := time.Parse(time.RFC3339, c.LastSeenAt)
|
||||||
|
if err != nil || now.Sub(seen) > CameraStaleAfter {
|
||||||
|
c.State = CameraStale
|
||||||
|
c.StateNote = "The shop computer has stopped reporting this camera. " +
|
||||||
|
"Check that the computer is on and Behavision is running on it."
|
||||||
|
c.Connected = nil
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if c.Connected != nil && *c.Connected {
|
||||||
|
c.State = CameraConnected
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.State = CameraNotConnect
|
||||||
|
c.StateNote = "The shop computer cannot open this camera's stream. " +
|
||||||
|
"Check the address, the password and the cabling."
|
||||||
|
}
|
||||||
|
|||||||
@@ -59,6 +59,11 @@ func scanCamera(row pgx.Row) (api.Camera, error) {
|
|||||||
// The KEY travels in ImageKey, which is json:"-", and the handler swaps it
|
// The KEY travels in ImageKey, which is json:"-", and the handler swaps it
|
||||||
// for a signed link. Same rule as an arrival's face.
|
// for a signed link. Same rule as an arrival's face.
|
||||||
c.Snapshot.Key = snapKey
|
c.Snapshot.Key = snapKey
|
||||||
|
// Here rather than in a handler, because every camera anybody reads comes
|
||||||
|
// through this function and a state computed per caller is a state one
|
||||||
|
// caller forgets - which is how a stale `connected` reached three screens
|
||||||
|
// at once.
|
||||||
|
c.CameraState(time.Now())
|
||||||
return c, nil
|
return c, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
2
server/internal/web/dist/index.html
vendored
2
server/internal/web/dist/index.html
vendored
@@ -6,7 +6,7 @@
|
|||||||
<meta name="color-scheme" content="dark" />
|
<meta name="color-scheme" content="dark" />
|
||||||
<link rel="icon" type="image/png" href="/favicon.png" />
|
<link rel="icon" type="image/png" href="/favicon.png" />
|
||||||
<title>Behavision</title>
|
<title>Behavision</title>
|
||||||
<script type="module" crossorigin src="/assets/index-Ckr5hGZd.js"></script>
|
<script type="module" crossorigin src="/assets/index-BuVkBNdU.js"></script>
|
||||||
<link rel="stylesheet" crossorigin href="/assets/index-D4KGRSVS.css">
|
<link rel="stylesheet" crossorigin href="/assets/index-D4KGRSVS.css">
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
|
|||||||
@@ -1,7 +1,32 @@
|
|||||||
"""cv2.FaceDetectorYN caches its input size and is not thread-safe, so camera
|
"""cv2.FaceDetectorYN caches its input size and is not thread-safe, so camera
|
||||||
workers must not share one. Skipped when the model is absent, matching the
|
workers must not share one. Skipped when the model is absent, matching the
|
||||||
faiss-optional pattern in test_index.py — the suite stays runnable with no
|
faiss-optional pattern in test_index.py — the suite stays runnable with no
|
||||||
models installed."""
|
models installed.
|
||||||
|
|
||||||
|
The premise is proved in a SUBPROCESS, and that is the whole lesson of this
|
||||||
|
file. The first version raced a shared detector in-process and asserted that
|
||||||
|
an exception came back, because on OpenCV 4.11 one usually did. It is a data
|
||||||
|
race in C++: what it produces is undefined, and on 4.14 what it mostly
|
||||||
|
produces is a segmentation fault. Measured, nine runs each, same machine:
|
||||||
|
|
||||||
|
numpy 1.26 / cv2 4.11 9 passed, 0 crashed
|
||||||
|
numpy 2.0 / cv2 4.11 8 passed, 1 crashed
|
||||||
|
numpy 1.26 / cv2 4.14 3 passed, 6 crashed
|
||||||
|
numpy 2.0 / cv2 4.14 2 passed, 7 crashed
|
||||||
|
|
||||||
|
numpy is not the variable; OpenCV is, and 4.11 crashing once says the hazard
|
||||||
|
was always there and 4.11 merely survived it. A test that takes the whole
|
||||||
|
suite down two runs in three is worse than no test: it turns "we upgraded
|
||||||
|
OpenCV" into a CI failure with no failing assertion in it, which is the
|
||||||
|
hardest kind to read.
|
||||||
|
|
||||||
|
None of this reaches the product. `Engine._build_worker` constructs a
|
||||||
|
FaceDetector per camera, which is what the rule says and what the second test
|
||||||
|
here guards.
|
||||||
|
"""
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import textwrap
|
||||||
import threading
|
import threading
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -11,10 +36,16 @@ import pytest
|
|||||||
from behavision.config import Config
|
from behavision.config import Config
|
||||||
from behavision.detection import YUNET_FILENAME, FaceDetector
|
from behavision.detection import YUNET_FILENAME, FaceDetector
|
||||||
|
|
||||||
MODELS = Path(__file__).resolve().parent.parent / "models"
|
ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
MODELS = ROOT / "models"
|
||||||
pytestmark = pytest.mark.skipif(not (MODELS / YUNET_FILENAME).exists(),
|
pytestmark = pytest.mark.skipif(not (MODELS / YUNET_FILENAME).exists(),
|
||||||
reason="YuNet model not installed")
|
reason="YuNet model not installed")
|
||||||
|
|
||||||
|
# A shared detector fails probabilistically, so one clean attempt proves
|
||||||
|
# nothing. Several do: the premise holds if ANY attempt misbehaves, and only
|
||||||
|
# an unbroken run of clean ones is evidence it has stopped being true.
|
||||||
|
ATTEMPTS = 6
|
||||||
|
|
||||||
|
|
||||||
def _detector():
|
def _detector():
|
||||||
d = Config().detection
|
d = Config().detection
|
||||||
@@ -44,11 +75,37 @@ def _race(det_a, det_b):
|
|||||||
return errors
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
_SHARED_RACE = textwrap.dedent("""
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, {root!r})
|
||||||
|
from tests.test_detector_concurrency import _detector, _race
|
||||||
|
shared = _detector()
|
||||||
|
print("raced" if _race(shared, shared) else "clean")
|
||||||
|
""")
|
||||||
|
|
||||||
|
|
||||||
|
def _shared_race_outcome():
|
||||||
|
"""Run one shared-detector race out of process.
|
||||||
|
|
||||||
|
Returns "raced" (an exception came back), "crashed" (the process died,
|
||||||
|
which is the same premise arriving by a blunter route) or "clean".
|
||||||
|
"""
|
||||||
|
proc = subprocess.run([sys.executable, "-c", _SHARED_RACE.format(root=str(ROOT))],
|
||||||
|
capture_output=True, text=True, timeout=120)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
return "crashed"
|
||||||
|
return proc.stdout.strip().splitlines()[-1] if proc.stdout.strip() else "clean"
|
||||||
|
|
||||||
|
|
||||||
def test_a_shared_detector_really_does_race():
|
def test_a_shared_detector_really_does_race():
|
||||||
"""Guards the premise: if this ever stops failing, the test below is
|
"""Guards the premise: if this ever stops failing, the test below is
|
||||||
proving nothing and the per-camera split can be revisited."""
|
proving nothing and the per-camera split can be revisited."""
|
||||||
shared = _detector()
|
seen = [_shared_race_outcome() for _ in range(ATTEMPTS)]
|
||||||
assert _race(shared, shared), "expected a shared detector to race"
|
assert any(o != "clean" for o in seen), (
|
||||||
|
f"a shared detector survived {ATTEMPTS} races ({seen}) - if that is "
|
||||||
|
"reproducible, cv2.FaceDetectorYN may have become thread-safe and the "
|
||||||
|
"per-camera rule in CLAUDE.md can be revisited"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_per_camera_detectors_do_not_race():
|
def test_per_camera_detectors_do_not_race():
|
||||||
|
|||||||
164
tests/test_model_download.py
Normal file
164
tests/test_model_download.py
Normal file
@@ -0,0 +1,164 @@
|
|||||||
|
"""Downloading the models is the last step of every fresh install, and it ran
|
||||||
|
into the one macOS trap nothing else here does.
|
||||||
|
|
||||||
|
A python.org macOS build ships its own OpenSSL with NO trust store, and
|
||||||
|
populates one only when somebody double-clicks `Install Certificates.command`
|
||||||
|
in the Python folder. Measured on a colleague's Mac: the engine installed
|
||||||
|
perfectly - numpy, onnxruntime, faiss, all of it - and then could not fetch a
|
||||||
|
230 KB model file, ending setup in forty lines of traceback about `_ssl.c`.
|
||||||
|
"""
|
||||||
|
import io
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import ssl
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from behavision import model_assets
|
||||||
|
|
||||||
|
|
||||||
|
class _Resp(io.BytesIO):
|
||||||
|
"""Enough of an http response for _fetch: read() and .headers."""
|
||||||
|
|
||||||
|
def __init__(self, payload: bytes):
|
||||||
|
super().__init__(payload)
|
||||||
|
self.headers = {"Content-Length": str(len(payload))}
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *exc):
|
||||||
|
self.close()
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _verify_error():
|
||||||
|
"""What urlopen ACTUALLY raises, which is not what it looks like.
|
||||||
|
|
||||||
|
urllib catches ssl.SSLCertVerificationError and re-raises
|
||||||
|
urllib.error.URLError(err), carrying the original on `.reason`. The first
|
||||||
|
version of this test raised the bare SSL error - a shape real urllib never
|
||||||
|
produces - so it passed against a fallback that could never fire, and the
|
||||||
|
fix shipped and failed on the machine it was written for with the exact
|
||||||
|
traceback it was meant to prevent.
|
||||||
|
"""
|
||||||
|
return urllib.error.URLError(ssl.SSLCertVerificationError(
|
||||||
|
"[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: "
|
||||||
|
"unable to get local issuer certificate"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_machine_with_no_trust_store_falls_back_to_the_bundled_one(monkeypatch):
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def fake(url, timeout=None, context=None):
|
||||||
|
calls.append(context)
|
||||||
|
if context is None:
|
||||||
|
raise _verify_error()
|
||||||
|
return _Resp(b"ok")
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", fake)
|
||||||
|
with model_assets._urlopen("https://example.invalid/m.onnx") as resp:
|
||||||
|
assert resp.read() == b"ok"
|
||||||
|
|
||||||
|
assert len(calls) == 2, f"expected a retry, got {calls}"
|
||||||
|
# Default FIRST, and that order is the point. On Windows and on a system
|
||||||
|
# Python the default context reads the machine's own certificate store,
|
||||||
|
# which is what makes a corporate proxy with its own root CA work.
|
||||||
|
# Replacing it unconditionally would break every site that has one in
|
||||||
|
# order to fix a different platform.
|
||||||
|
assert calls[0] is None
|
||||||
|
assert isinstance(calls[1], ssl.SSLContext)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_working_trust_store_is_used_as_is(monkeypatch):
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def fake(url, timeout=None, context=None):
|
||||||
|
calls.append(context)
|
||||||
|
return _Resp(b"ok")
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", fake)
|
||||||
|
model_assets._urlopen("https://example.invalid/m.onnx").close()
|
||||||
|
assert calls == [None], "the machine's own certificate store was bypassed"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_real_network_failure_is_not_disguised_as_a_certificate_problem(monkeypatch):
|
||||||
|
"""A URLError is not automatically a certificate problem.
|
||||||
|
|
||||||
|
"No route to host" and "connection refused" arrive as URLError too, and
|
||||||
|
retrying those with a different CA list changes nothing except how long
|
||||||
|
the operator waits for the real message.
|
||||||
|
"""
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def fake(url, timeout=None, context=None):
|
||||||
|
calls.append(context)
|
||||||
|
raise urllib.error.URLError("no route to host")
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", fake)
|
||||||
|
with pytest.raises(urllib.error.URLError):
|
||||||
|
model_assets._urlopen("https://example.invalid/m.onnx")
|
||||||
|
assert calls == [None], f"a dead network was retried as a CA problem: {calls}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_progress_lines_survive_the_rewrite(tmp_path, monkeypatch, caplog):
|
||||||
|
"""`download: <label> <n>%` is a CONTRACT, not logging.
|
||||||
|
|
||||||
|
The supervisor parses it (progressRe) to put first-run progress in the
|
||||||
|
tray and the window, because the API is not up yet and a shop PC showing
|
||||||
|
a stopped engine for five minutes after install looks broken. Switching
|
||||||
|
off urlretrieve - needed because it offers no way to pass an SSL context -
|
||||||
|
is exactly the kind of change that drops it silently.
|
||||||
|
"""
|
||||||
|
payload = b"x" * (64 * 1024 * 8)
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen",
|
||||||
|
lambda url, timeout=None, context=None: _Resp(payload))
|
||||||
|
dest = tmp_path / "model.onnx"
|
||||||
|
with caplog.at_level(logging.INFO, logger="behavision.model_assets"):
|
||||||
|
model_assets._fetch("https://example.invalid/m.onnx", dest, "face detector")
|
||||||
|
|
||||||
|
assert dest.read_bytes() == payload
|
||||||
|
lines = [r.getMessage() for r in caplog.records]
|
||||||
|
pct = [ln for ln in lines if ln.startswith("download: face detector ")]
|
||||||
|
assert pct, f"no progress lines at all: {lines}"
|
||||||
|
assert "download: face detector 100%" in pct, pct
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.skipif(not os.environ.get("BEHAVISION_NETWORK_TESTS"),
|
||||||
|
reason="set BEHAVISION_NETWORK_TESTS=1 to reach github.com")
|
||||||
|
def test_against_a_real_machine_with_no_trust_store(tmp_path, monkeypatch):
|
||||||
|
"""The whole thing, over the real network, with the real failure.
|
||||||
|
|
||||||
|
Every stub above is a statement about what I believe urllib does, and the
|
||||||
|
first version of this file proved how much that is worth. Python's default
|
||||||
|
context honours SSL_CERT_FILE, so pointing it at an EMPTY file reproduces
|
||||||
|
a python.org macOS build exactly: a default context that trusts nobody.
|
||||||
|
certifi is loaded by path and is unaffected by the variable, so if the
|
||||||
|
fallback works here it works there.
|
||||||
|
"""
|
||||||
|
empty = tmp_path / "no-cas.pem"
|
||||||
|
empty.write_text("")
|
||||||
|
monkeypatch.setenv("SSL_CERT_FILE", str(empty))
|
||||||
|
monkeypatch.setenv("SSL_CERT_DIR", str(tmp_path / "nothing"))
|
||||||
|
|
||||||
|
# Not every interpreter can be put into that state, and pretending
|
||||||
|
# otherwise would turn this into a test that passes by not running. A
|
||||||
|
# Python linked against LibreSSL - which is what macOS Command Line Tools
|
||||||
|
# ships - reads the system keychain and ignores SSL_CERT_FILE entirely:
|
||||||
|
# measured, 128 CAs loaded with the variable pointing at an empty file. A
|
||||||
|
# python.org build links OpenSSL and honours it: 0 CAs, which is the
|
||||||
|
# condition being reproduced.
|
||||||
|
if ssl.create_default_context().get_ca_certs():
|
||||||
|
pytest.skip("this interpreter ignores SSL_CERT_FILE (%s), so the "
|
||||||
|
"no-trust-store condition cannot be reproduced here"
|
||||||
|
% ssl.OPENSSL_VERSION)
|
||||||
|
|
||||||
|
# The premise: the default context really is broken in this process.
|
||||||
|
with pytest.raises(urllib.error.URLError) as caught:
|
||||||
|
urllib.request.urlopen(model_assets.YUNET_URL, timeout=30)
|
||||||
|
assert model_assets._is_cert_failure(caught.value), caught.value
|
||||||
|
|
||||||
|
dest = tmp_path / "yunet.onnx"
|
||||||
|
model_assets._fetch(model_assets.YUNET_URL, dest, "face detector")
|
||||||
|
assert dest.stat().st_size > 200_000
|
||||||
@@ -47,9 +47,13 @@ export default function Cameras({ user }) {
|
|||||||
|
|
||||||
const canEdit = ['admin', 'owner', 'manager'].includes(user.role)
|
const canEdit = ['admin', 'owner', 'manager'].includes(user.role)
|
||||||
const list = cams || []
|
const list = cams || []
|
||||||
const up = list.filter(c => c.connected).length
|
// Counted off `state`, the one field the server computes, never off
|
||||||
const down = list.filter(c => c.connected === false).length
|
// `connected`. Two places deciding the same fact is how a shop came out
|
||||||
const waiting = list.filter(c => c.connected == null).length
|
// labelled Working, in green, above "2 of 3 cameras not connecting".
|
||||||
|
const up = list.filter(c => c.state === 'connected').length
|
||||||
|
const down = list.filter(c => c.state === 'not_connecting').length
|
||||||
|
const waiting = list.filter(c => c.state === 'waiting').length
|
||||||
|
const stale = list.filter(c => c.state === 'stale').length
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<>
|
<>
|
||||||
@@ -58,6 +62,7 @@ export default function Cameras({ user }) {
|
|||||||
<p className="sub">
|
<p className="sub">
|
||||||
{list.length} {list.length === 1 ? 'camera' : 'cameras'}
|
{list.length} {list.length === 1 ? 'camera' : 'cameras'}
|
||||||
{up > 0 && <> · <b className="ok">{up} connected</b></>}
|
{up > 0 && <> · <b className="ok">{up} connected</b></>}
|
||||||
|
{stale > 0 && <> · <b className="warn">{stale} not reporting</b></>}
|
||||||
{down > 0 && <> · <b className="bad">{down} down</b></>}
|
{down > 0 && <> · <b className="bad">{down} down</b></>}
|
||||||
{waiting > 0 && <> · {waiting} waiting for the shop PC</>}
|
{waiting > 0 && <> · {waiting} waiting for the shop PC</>}
|
||||||
</p>
|
</p>
|
||||||
@@ -108,10 +113,13 @@ function CameraCard({ cam, canEdit, onEdit, onWatch }) {
|
|||||||
// Three states, not two. A camera nobody has tried yet is not a camera that
|
// Three states, not two. A camera nobody has tried yet is not a camera that
|
||||||
// is down, and telling an operator to check the cabling on a camera the shop
|
// is down, and telling an operator to check the cabling on a camera the shop
|
||||||
// PC has not even seen sends them to the wrong building.
|
// PC has not even seen sends them to the wrong building.
|
||||||
const state = cam.connected == null ? 'idle'
|
// Four states, and the fourth is the one that was missing: a camera whose
|
||||||
: cam.connected ? 'ok' : 'bad'
|
// shop PC has stopped reporting it. The agent correctly says nothing when it
|
||||||
const words = cam.connected == null ? 'Waiting for the shop PC'
|
// cannot reach the engine, so the last value it sent used to sit in the
|
||||||
: cam.connected ? 'Connected' : 'Not connecting'
|
// database reading Connected - measured at 34 minutes on the live estate.
|
||||||
|
const state = { connected: 'ok', not_connecting: 'bad', stale: 'warn' }[cam.state] || 'idle'
|
||||||
|
const words = { connected: 'Connected', not_connecting: 'Not connecting',
|
||||||
|
stale: 'Not reporting' }[cam.state] || 'Waiting for the shop PC'
|
||||||
const verified = verification(cam)
|
const verified = verification(cam)
|
||||||
|
|
||||||
return (
|
return (
|
||||||
|
|||||||
Reference in New Issue
Block a user