Compare commits
3 Commits
v0.5.2-dem
...
v0.5.5-dem
| Author | SHA1 | Date | |
|---|---|---|---|
| 3cddd9c2e1 | |||
| 248025cdf9 | |||
| ff4f95c3b0 |
163
CLAUDE.md
163
CLAUDE.md
@@ -3436,3 +3436,166 @@ three, and `api.CameraState` is the one function that decides them:
|
|||||||
- **An unparseable `last_seen_at` is stale**, not connected. It should be
|
- **An unparseable `last_seen_at` is stale**, not connected. It should be
|
||||||
impossible, which is precisely why it must not fall through to the state that
|
impossible, which is precisely why it must not fall through to the state that
|
||||||
says everything is fine.
|
says everything is fine.
|
||||||
|
|
||||||
|
## A demo on somebody else's Mac found four things, all of them silent
|
||||||
|
|
||||||
|
Three failures in one afternoon on a colleague's machine, plus one the fixing
|
||||||
|
uncovered. Every one produced a message that was true and useless.
|
||||||
|
|
||||||
|
### behavision-setup chose the Python least likely to work
|
||||||
|
|
||||||
|
`findPython` walked `3.14, 3.13, 3.12, 3.11, 3.10` and took the first hit — a
|
||||||
|
floor with **no ceiling**, which is exactly backwards. The newest Python on a
|
||||||
|
machine is the one least likely to have binary wheels for anything. It picked
|
||||||
|
3.14, pip found no numpy wheel for cp314 (`numpy<2.0` caps the resolver at
|
||||||
|
1.26.4, whose newest is cp312), fell back to building numpy from source and
|
||||||
|
produced `ERROR: Unknown compiler(s)`; once the operator had installed Xcode's
|
||||||
|
command line tools to get past that, ten minutes of compiling ended in
|
||||||
|
`<arm_neon.h> is intended only for ARM and AArch64 targets`.
|
||||||
|
|
||||||
|
Two screens of C compiler output on a shop counter, for a version choice this
|
||||||
|
program made silently. `maxMinor` refuses in one line before anything is
|
||||||
|
downloaded, and **"too new" is a different message from "too old"** — telling
|
||||||
|
somebody holding Python 3.14 that no Python was found sends them to install a
|
||||||
|
newer one, which is the direction that just failed. It is a *wheel-availability*
|
||||||
|
ceiling, not a language one: onnxruntime is the binding dependency today
|
||||||
|
(cp314 is its newest), numpy publishes further ahead, and opencv ships a
|
||||||
|
stable-ABI wheel that covers everything.
|
||||||
|
|
||||||
|
### `numpy<2.0` was the cap; OpenCV was the hazard
|
||||||
|
|
||||||
|
Widening to `<3.0` needed proof, and the proof found something else. Nine runs
|
||||||
|
of the detector guard per combination, one machine, one sitting:
|
||||||
|
|
||||||
|
```
|
||||||
|
numpy 1.26 / cv2 4.11 9 passed, 0 crashed
|
||||||
|
numpy 2.0 / cv2 4.11 8 passed, 1 crashed
|
||||||
|
numpy 1.26 / cv2 4.14 3 passed, 6 crashed
|
||||||
|
numpy 2.0 / cv2 4.14 2 passed, 7 crashed
|
||||||
|
```
|
||||||
|
|
||||||
|
**numpy is not the variable; OpenCV is** — the third row is numpy 1.26. The
|
||||||
|
crash was `test_a_shared_detector_really_does_race`, which races a shared
|
||||||
|
`cv2.FaceDetectorYN` on purpose to prove the per-camera rule. That is undefined
|
||||||
|
behaviour in C++: 4.11 usually turned it into an exception, 4.14 usually turns
|
||||||
|
it into a **segfault**, and 4.11 crashing once says the hazard was always there
|
||||||
|
and 4.11 merely survived it.
|
||||||
|
|
||||||
|
It never reached the product — `Engine._build_worker` builds a detector per
|
||||||
|
camera, which is the rule and is what the second test guards. What it reached
|
||||||
|
was the suite: two runs in three died with **no failing assertion in them**,
|
||||||
|
turning "we upgraded OpenCV" into the hardest kind of CI failure to read. The
|
||||||
|
race now runs in a **subprocess**, so a segfault is an observed outcome rather
|
||||||
|
than the end of the run, and one clean attempt proves nothing — the premise
|
||||||
|
holds if *any* of several attempts misbehaves. With that fixed the suite is
|
||||||
|
226 passed / 2 skipped on numpy 2.0.2, five runs out of five.
|
||||||
|
|
||||||
|
`opencv-python` stays capped below 5. Everything above was measured on 4.x, and
|
||||||
|
an uncapped `>=4.8.1` means every NEW install silently gets a major release
|
||||||
|
this project has never run a real camera through while every existing one keeps
|
||||||
|
4.11.
|
||||||
|
|
||||||
|
### And the fix was defeated by the wreckage of the bug
|
||||||
|
|
||||||
|
`makeVenv` reused any environment already on disk, whatever Python built it.
|
||||||
|
That machine had a runtime built by **3.14**, left behind by the run that
|
||||||
|
failed — so with the ceiling in place setup would choose a good interpreter,
|
||||||
|
reach `makeVenv`, find the 3.14 environment, keep it, and die in the same clang
|
||||||
|
error as before. A fix a user cannot reach because the bug's own debris is in
|
||||||
|
the way is not a fix, and it would have read as the release not working.
|
||||||
|
|
||||||
|
It now asks the interpreter inside an existing environment what it is and
|
||||||
|
rebuilds when the answer is unsupported, saying so. Rebuilding costs a
|
||||||
|
re-download of the libraries and nothing else — the models live in the state
|
||||||
|
root, not in there. An environment that cannot be asked counts as unusable
|
||||||
|
too: a half-created one answers nothing, and reusing it fails later in pip
|
||||||
|
with an error about a package rather than about the environment.
|
||||||
|
|
||||||
|
### One MQTT client id for a whole shop, so two PCs fought over it
|
||||||
|
|
||||||
|
`behavision-<client>-<site>` is the same string on every computer claimed to
|
||||||
|
one site. MQTT requires client ids to be unique and a broker enforces it by
|
||||||
|
disconnecting the older session when a new one arrives with the same id, so the
|
||||||
|
colleague's Mac and the shop's own till took turns kicking each other off:
|
||||||
|
|
||||||
|
```
|
||||||
|
broker connected / broker connection lost: EOF / broker connected / EOF / ...
|
||||||
|
```
|
||||||
|
|
||||||
|
**The damage is not confined to the new machine.** The shop's till is the other
|
||||||
|
half of that loop, so somebody signing in on a laptop to look at the product
|
||||||
|
stops a live shop delivering visits — and from each end it reads as an unstable
|
||||||
|
network, because nothing says otherwise.
|
||||||
|
|
||||||
|
`Config.MQTTClientID()` appends a per-installation id, minted on first load and
|
||||||
|
written back so an existing install gets one without anybody doing anything.
|
||||||
|
The site stays in the name because that is what a broker log is read *by*. A
|
||||||
|
config that could not be written falls back to a per-run id rather than a
|
||||||
|
shared one: the right failure is a new name in the log after a restart, not the
|
||||||
|
collision this exists to end.
|
||||||
|
|
||||||
|
### "no such file or directory" for an engine nobody had installed
|
||||||
|
|
||||||
|
Pressing Start with no engine went straight to the supervisor, which reported
|
||||||
|
what `exec` reported:
|
||||||
|
|
||||||
|
```
|
||||||
|
engine failed to start: fork/exec /private/var/folders/c2/.../AppTranslocation/
|
||||||
|
500A5354-.../d/Behavision.app/Contents/MacOS/engine/behavision:
|
||||||
|
no such file or directory
|
||||||
|
```
|
||||||
|
|
||||||
|
Every word true, none of it saying *run the setup tool*. The startup path did
|
||||||
|
have that sentence — in a log file nobody on a shop counter opens.
|
||||||
|
`App.engineMissing()` is now the one function the startup path, the Start
|
||||||
|
button and the status panel all consult, so three surfaces cannot give three
|
||||||
|
accounts of one fact.
|
||||||
|
|
||||||
|
It also names **App Translocation**, which is in that path and is unguessable.
|
||||||
|
macOS quarantines a downloaded app it cannot verify and runs it from a randomly
|
||||||
|
named read-only copy, so every relative path resolves inside that copy — which
|
||||||
|
is why the engine folder appears missing from a bundle that plainly contains
|
||||||
|
one, and why installing into it would not survive a restart. Fixed by dragging
|
||||||
|
the app to Applications; saying nothing leaves somebody re-running a setup tool
|
||||||
|
that cannot win. The product is unsigned, so this is the *normal* first-run
|
||||||
|
state on every Mac, not an edge case.
|
||||||
|
|
||||||
|
### And then it could not download a 230 KB file
|
||||||
|
|
||||||
|
With all of the above fixed the install succeeded on that Mac - Python 3.14
|
||||||
|
chosen and accepted, numpy 2.5.3, onnxruntime 1.30, faiss 1.15.1, the engine
|
||||||
|
itself - and setup died on the last step, fetching the YuNet model:
|
||||||
|
|
||||||
|
```
|
||||||
|
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
|
||||||
|
certificate verify failed: unable to get local issuer certificate
|
||||||
|
```
|
||||||
|
|
||||||
|
A python.org macOS build ships its **own OpenSSL with no trust store**, and
|
||||||
|
populates one only when somebody double-clicks `Install Certificates.command`
|
||||||
|
in the Python folder. Nobody installing face-recognition software has any
|
||||||
|
reason to know that exists, and the failure is forty lines of traceback about
|
||||||
|
`_ssl.c` at the end of a ten-minute install.
|
||||||
|
|
||||||
|
`_urlopen` tries the default context first and retries with **certifi's**
|
||||||
|
bundle on a verification failure. The order is the whole design:
|
||||||
|
|
||||||
|
- Default first, because on Windows and on a system or Homebrew Python the
|
||||||
|
default context reads the machine's own certificate store - which is what
|
||||||
|
makes a corporate proxy with its own root CA work. Replacing it
|
||||||
|
unconditionally would break every site that has one in order to fix a
|
||||||
|
different platform.
|
||||||
|
- certifi second, because it is already installed: `requests` is a hard
|
||||||
|
dependency and brings it.
|
||||||
|
- A `URLError` is re-raised untouched. "No route to host" and "no trust store"
|
||||||
|
are different problems, and retrying the first with a different CA list only
|
||||||
|
delays the real message.
|
||||||
|
|
||||||
|
`urlretrieve` had to go, since it offers no way to pass a context - and that is
|
||||||
|
exactly the kind of rewrite that silently drops something. The
|
||||||
|
`download: <label> <n>%` lines are a **contract**: `supervisor.go`'s
|
||||||
|
`progressRe` parses them to put first-run progress in the tray, because the API
|
||||||
|
is not up yet and a shop PC showing a stopped engine for five minutes after
|
||||||
|
install looks broken. `tests/test_model_download.py` asserts them, and the
|
||||||
|
rewritten fetch was checked against the real URL: 232,589 bytes, sha256
|
||||||
|
identical to the model already on disk.
|
||||||
|
|||||||
@@ -46,6 +46,61 @@ import (
|
|||||||
// otherwise arrive as a syntax error deep inside a dependency.
|
// otherwise arrive as a syntax error deep inside a dependency.
|
||||||
const minMinor = 10
|
const minMinor = 10
|
||||||
|
|
||||||
|
// maxMinor is a WHEEL-availability ceiling, not a language one, and it is the
|
||||||
|
// reason this constant exists at all.
|
||||||
|
//
|
||||||
|
// findPython used to take the newest interpreter it could find, with a floor
|
||||||
|
// and no ceiling - which is precisely backwards, because the newest Python is
|
||||||
|
// the one least likely to have binary wheels for anything. Measured on a
|
||||||
|
// second Mac: it chose Python 3.14, pip found no numpy wheel for cp314, fell
|
||||||
|
// back to building numpy from source, and produced
|
||||||
|
//
|
||||||
|
// ERROR: Unknown compiler(s): [['cc'], ['gcc'], ['clang'], ...]
|
||||||
|
//
|
||||||
|
// then, once the operator installed Xcode's command line tools to get past
|
||||||
|
// that, ten minutes of compiling ending in
|
||||||
|
//
|
||||||
|
// arm_neon.h:28:2: error: "<arm_neon.h> is intended only for ARM and
|
||||||
|
// AArch64 targets"
|
||||||
|
//
|
||||||
|
// Two screens of C compiler output, on a shop counter, for a version choice
|
||||||
|
// made silently by this program. Refusing in one line, before anything is
|
||||||
|
// downloaded, is the whole of the fix.
|
||||||
|
//
|
||||||
|
// Raise it when the dependency set has wheels for the next version. Today
|
||||||
|
// onnxruntime is the binding one (cp314 is its newest); numpy publishes
|
||||||
|
// further ahead, and opencv-python ships a stable-ABI wheel that covers
|
||||||
|
// everything. `pip download --only-binary=:all: -r requirements.txt` against
|
||||||
|
// a candidate interpreter is the check.
|
||||||
|
const maxMinor = 14
|
||||||
|
|
||||||
|
// The three answers a candidate interpreter can get. Three, not two: a
|
||||||
|
// version that is too new and one that is too old need opposite actions from
|
||||||
|
// the operator, and collapsing them tells somebody holding Python 3.14 to go
|
||||||
|
// and install a newer Python.
|
||||||
|
const (
|
||||||
|
verdictOK = "ok"
|
||||||
|
verdictTooOld = "old"
|
||||||
|
verdictTooNew = "new"
|
||||||
|
verdictUnknown = "unparseable"
|
||||||
|
)
|
||||||
|
|
||||||
|
func pythonVerdict(major, minor int, parsed bool) string {
|
||||||
|
switch {
|
||||||
|
case !parsed:
|
||||||
|
return verdictUnknown
|
||||||
|
case major != 3:
|
||||||
|
// Python 4 is not a version this has been tried against, and 2 is
|
||||||
|
// long gone. Neither is a thing to guess about.
|
||||||
|
return verdictTooNew
|
||||||
|
case minor < minMinor:
|
||||||
|
return verdictTooOld
|
||||||
|
case minor > maxMinor:
|
||||||
|
return verdictTooNew
|
||||||
|
}
|
||||||
|
return verdictOK
|
||||||
|
}
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
if err := run(); err != nil {
|
if err := run(); err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "\n Setup did not finish: %v\n\n", err)
|
fmt.Fprintf(os.Stderr, "\n Setup did not finish: %v\n\n", err)
|
||||||
@@ -267,7 +322,13 @@ func findPython() (string, string, error) {
|
|||||||
// `python3` therefore told a Mac with Python 3.12 sitting on it to go and
|
// `python3` therefore told a Mac with Python 3.12 sitting on it to go and
|
||||||
// install Python - measured on this machine, which has 3.12 under
|
// install Python - measured on this machine, which has 3.12 under
|
||||||
// ~/.local/opt and reported "Found, but too old: python3 3.9".
|
// ~/.local/opt and reported "Found, but too old: python3 3.9".
|
||||||
versions := []string{"3.14", "3.13", "3.12", "3.11", "3.10"}
|
// Newest first WITHIN the supported range. Newest overall is what broke
|
||||||
|
// this; a version nobody has built wheels for is not a better choice than
|
||||||
|
// one that works.
|
||||||
|
var versions []string
|
||||||
|
for v := maxMinor; v >= minMinor; v-- {
|
||||||
|
versions = append(versions, fmt.Sprintf("3.%d", v))
|
||||||
|
}
|
||||||
for _, v := range versions {
|
for _, v := range versions {
|
||||||
cands = append(cands, cand{"python" + v, nil})
|
cands = append(cands, cand{"python" + v, nil})
|
||||||
}
|
}
|
||||||
@@ -290,7 +351,7 @@ func findPython() (string, string, error) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var tried []string
|
var tried, tooNew []string
|
||||||
for _, c := range cands {
|
for _, c := range cands {
|
||||||
exe := c.exe
|
exe := c.exe
|
||||||
if filepath.IsAbs(exe) {
|
if filepath.IsAbs(exe) {
|
||||||
@@ -314,7 +375,18 @@ func findPython() (string, string, error) {
|
|||||||
}
|
}
|
||||||
ver := strings.TrimSpace(string(out))
|
ver := strings.TrimSpace(string(out))
|
||||||
tried = append(tried, c.exe+" "+ver)
|
tried = append(tried, c.exe+" "+ver)
|
||||||
if major, minor, ok := parseVer(ver); ok && (major > 3 || (major == 3 && minor >= minMinor)) {
|
major, minor, parsed := parseVer(ver)
|
||||||
|
switch verdict := pythonVerdict(major, minor, parsed); verdict {
|
||||||
|
case verdictTooNew:
|
||||||
|
// Recorded separately: "too new" and "too old" need opposite
|
||||||
|
// actions, and a single "found, but unsuitable" list sends
|
||||||
|
// somebody to upgrade a Python that is already past the problem.
|
||||||
|
tooNew = append(tooNew, c.exe+" "+ver)
|
||||||
|
continue
|
||||||
|
case verdictTooOld, verdictUnknown:
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
{
|
||||||
full := exe
|
full := exe
|
||||||
if len(c.args) > 0 {
|
if len(c.args) > 0 {
|
||||||
full = exe + " " + strings.Join(c.args, " ")
|
full = exe + " " + strings.Join(c.args, " ")
|
||||||
@@ -327,7 +399,30 @@ func findPython() (string, string, error) {
|
|||||||
// python.exe to PATH" on a Windows installer page reads as software that
|
// python.exe to PATH" on a Windows installer page reads as software that
|
||||||
// does not know where it is running, which is exactly the moment somebody
|
// does not know where it is running, which is exactly the moment somebody
|
||||||
// stops trusting the rest of what it says.
|
// stops trusting the rest of what it says.
|
||||||
msg := "no Python 3.10 or newer was found on this computer.\n\n"
|
// Only a too-new Python is a different problem with a different fix, and
|
||||||
|
// saying "no Python was found" to somebody looking at Python 3.14 is the
|
||||||
|
// kind of message that makes people stop believing the next one.
|
||||||
|
if len(tooNew) > 0 && len(tried) == 0 {
|
||||||
|
// Built as a value and wrapped, not written as an fmt.Errorf literal:
|
||||||
|
// this is a paragraph shown to an operator, and a linter that wants
|
||||||
|
// error strings to be lower-case fragments is right about errors
|
||||||
|
// programs read and wrong about the ones people do.
|
||||||
|
tooNewMsg := fmt.Sprintf(
|
||||||
|
"this computer has %s, which is newer than Behavision supports.\n\n"+
|
||||||
|
" Some of the libraries the engine needs have no build for it\n"+
|
||||||
|
" yet, so installing would fail part-way through.\n\n"+
|
||||||
|
" Install Python 3.%d and run this again:\n"+
|
||||||
|
" macOS: brew install python@3.%d\n"+
|
||||||
|
" or https://www.python.org/downloads/macos/\n"+
|
||||||
|
" Windows: https://www.python.org/downloads/windows/\n\n"+
|
||||||
|
" Both versions can sit on the machine together; this picks\n"+
|
||||||
|
" the one it can use.",
|
||||||
|
strings.Join(tooNew, ", "), maxMinor, maxMinor)
|
||||||
|
return "", "", errors.New(tooNewMsg)
|
||||||
|
}
|
||||||
|
|
||||||
|
msg := fmt.Sprintf("no Python between 3.%d and 3.%d was found on this computer.\n\n",
|
||||||
|
minMinor, maxMinor)
|
||||||
if runtime.GOOS == "windows" {
|
if runtime.GOOS == "windows" {
|
||||||
msg += " Install it from https://www.python.org/downloads/windows/\n" +
|
msg += " Install it from https://www.python.org/downloads/windows/\n" +
|
||||||
" and tick \"Add python.exe to PATH\" on the first screen,\n" +
|
" and tick \"Add python.exe to PATH\" on the first screen,\n" +
|
||||||
@@ -339,6 +434,9 @@ func findPython() (string, string, error) {
|
|||||||
if len(tried) > 0 {
|
if len(tried) > 0 {
|
||||||
msg += "\n\n Found, but too old: " + strings.Join(tried, ", ")
|
msg += "\n\n Found, but too old: " + strings.Join(tried, ", ")
|
||||||
}
|
}
|
||||||
|
if len(tooNew) > 0 {
|
||||||
|
msg += "\n\n Found, but too new: " + strings.Join(tooNew, ", ")
|
||||||
|
}
|
||||||
return "", "", errors.New(msg)
|
return "", "", errors.New(msg)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -375,14 +473,52 @@ func venvPython(venv string) string {
|
|||||||
// engine requires - inside a shared interpreter is how you break the other
|
// engine requires - inside a shared interpreter is how you break the other
|
||||||
// thing months later, silently.
|
// thing months later, silently.
|
||||||
func makeVenv(py, venv string) error {
|
func makeVenv(py, venv string) error {
|
||||||
|
// An existing environment is reused - but only if the Python inside it is
|
||||||
|
// one this build supports.
|
||||||
|
//
|
||||||
|
// It used to be reused unconditionally, and that would have made the
|
||||||
|
// version ceiling above look like it did not work. The machine this was
|
||||||
|
// all found on already had a runtime built by Python 3.14, from the run
|
||||||
|
// that failed: with the ceiling in place setup would choose a good
|
||||||
|
// interpreter, reach here, find the 3.14 environment, keep it, and die in
|
||||||
|
// the same clang error as before. A fix that is defeated by the wreckage
|
||||||
|
// of the bug it fixes is not one.
|
||||||
|
//
|
||||||
|
// Rebuilding costs a re-download of the libraries and nothing else. The
|
||||||
|
// models are in the state root, not in here, so they survive.
|
||||||
if _, err := os.Stat(venvPython(venv)); err == nil {
|
if _, err := os.Stat(venvPython(venv)); err == nil {
|
||||||
return nil // already built; pip below brings it up to date
|
ok, ver := venvUsable(venv)
|
||||||
|
if ok {
|
||||||
|
return nil // pip below brings it up to date
|
||||||
|
}
|
||||||
|
fmt.Printf(" [..] %-24s %s\n", "Rebuilding environment",
|
||||||
|
"the existing one uses "+ver+", which is not supported")
|
||||||
|
if err := os.RemoveAll(venv); err != nil {
|
||||||
|
return fmt.Errorf("removing the old environment at %s: %w", venv, err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
exe, args := splitLauncher(py)
|
exe, args := splitLauncher(py)
|
||||||
args = append(args, "-m", "venv", venv)
|
args = append(args, "-m", "venv", venv)
|
||||||
return stream(exec.Command(exe, args...), "creating the virtual environment")
|
return stream(exec.Command(exe, args...), "creating the virtual environment")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// venvUsable reports whether the interpreter already inside an environment is
|
||||||
|
// one this build supports, and what it is when it is not.
|
||||||
|
//
|
||||||
|
// An environment that cannot be asked counts as unusable: a half-created or
|
||||||
|
// truncated one answers nothing, and reusing it fails later in pip with an
|
||||||
|
// error about a package rather than about the environment.
|
||||||
|
func venvUsable(venv string) (bool, string) {
|
||||||
|
out, err := exec.Command(venvPython(venv), "-c",
|
||||||
|
"import sys;print('%d.%d'%sys.version_info[:2])").Output()
|
||||||
|
if err != nil {
|
||||||
|
return false, "an interpreter that will not run"
|
||||||
|
}
|
||||||
|
ver := strings.TrimSpace(string(out))
|
||||||
|
major, minor, parsed := parseVer(ver)
|
||||||
|
return pythonVerdict(major, minor, parsed) == verdictOK, "Python " + ver
|
||||||
|
}
|
||||||
|
|
||||||
func pipInstall(vpy, src string) error {
|
func pipInstall(vpy, src string) error {
|
||||||
fmt.Println(" Installing the engine and its libraries. This downloads a few")
|
fmt.Println(" Installing the engine and its libraries. This downloads a few")
|
||||||
fmt.Println(" hundred megabytes and takes a while on a slow connection.")
|
fmt.Println(" hundred megabytes and takes a while on a slow connection.")
|
||||||
|
|||||||
112
agent/cmd/behavision-setup/python_test.go
Normal file
112
agent/cmd/behavision-setup/python_test.go
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The choice this program makes silently, and got wrong.
|
||||||
|
//
|
||||||
|
// findPython took the newest interpreter on the machine, with a floor and no
|
||||||
|
// ceiling - backwards, because the newest Python is the one least likely to
|
||||||
|
// have binary wheels. On a Mac holding Python 3.14 it chose 3.14, pip found
|
||||||
|
// no numpy wheel for cp314, fell back to a source build and produced two
|
||||||
|
// screens of clang errors ending in "<arm_neon.h> is intended only for ARM
|
||||||
|
// and AArch64 targets". The operator's machine was fine; the version was not.
|
||||||
|
func TestTooNewIsRefusedRatherThanCompiled(t *testing.T) {
|
||||||
|
if got := pythonVerdict(3, maxMinor+1, true); got != verdictTooNew {
|
||||||
|
t.Errorf("3.%d = %q, want %q - picking it means a source build",
|
||||||
|
maxMinor+1, got, verdictTooNew)
|
||||||
|
}
|
||||||
|
if got := pythonVerdict(3, maxMinor, true); got != verdictOK {
|
||||||
|
t.Errorf("3.%d = %q, want %q - the ceiling is inclusive", maxMinor, got, verdictOK)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Too old and too new must stay different answers. Telling somebody holding
|
||||||
|
// Python 3.14 that no Python was found, or that theirs is too old, sends them
|
||||||
|
// to install a newer one - which is the direction that already failed.
|
||||||
|
func TestOldAndNewAreDifferentAnswers(t *testing.T) {
|
||||||
|
old := pythonVerdict(3, minMinor-1, true)
|
||||||
|
fresh := pythonVerdict(3, maxMinor+1, true)
|
||||||
|
if old == fresh {
|
||||||
|
t.Fatalf("3.%d and 3.%d both reported %q", minMinor-1, maxMinor+1, old)
|
||||||
|
}
|
||||||
|
if old != verdictTooOld {
|
||||||
|
t.Errorf("3.%d = %q, want %q", minMinor-1, old, verdictTooOld)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every version in the range is accepted, so the window this program claims
|
||||||
|
// to support is the one it actually uses.
|
||||||
|
func TestTheWholeSupportedRangeIsAccepted(t *testing.T) {
|
||||||
|
for m := minMinor; m <= maxMinor; m++ {
|
||||||
|
if got := pythonVerdict(3, m, true); got != verdictOK {
|
||||||
|
t.Errorf("3.%d = %q, want %q", m, got, verdictOK)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if minMinor > maxMinor {
|
||||||
|
t.Fatal("the supported range is empty; nothing would ever be chosen")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A major version nobody has tested against is not something to guess at, and
|
||||||
|
// an unreadable version string is not a working interpreter.
|
||||||
|
func TestUnknownVersionsAreNotAccepted(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
name string
|
||||||
|
major, minor int
|
||||||
|
parsed bool
|
||||||
|
}{
|
||||||
|
{"python 4", 4, 0, true},
|
||||||
|
{"python 2", 2, 7, true},
|
||||||
|
{"unparseable", 0, 0, false},
|
||||||
|
} {
|
||||||
|
if got := pythonVerdict(c.major, c.minor, c.parsed); got == verdictOK {
|
||||||
|
t.Errorf("%s was accepted", c.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An environment already on disk is reused, and that is right until the Python
|
||||||
|
// inside it is one this build cannot use.
|
||||||
|
//
|
||||||
|
// It was reused unconditionally, which would have defeated the ceiling above
|
||||||
|
// on the exact machine that found the bug: that Mac already had a runtime
|
||||||
|
// built by Python 3.14, left behind by the run that failed. Setup would pick a
|
||||||
|
// good interpreter, find the 3.14 environment, keep it, and die in the same
|
||||||
|
// clang error as before - a fix defeated by the wreckage of the bug it fixes.
|
||||||
|
//
|
||||||
|
// Real environments, not a fake: the thing under test is what an interpreter
|
||||||
|
// on disk reports about itself.
|
||||||
|
func TestAnUnsupportedEnvironmentIsNotReused(t *testing.T) {
|
||||||
|
py, _, err := findPython()
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("no supported Python on this machine: %v", err)
|
||||||
|
}
|
||||||
|
venv := filepath.Join(t.TempDir(), "runtime")
|
||||||
|
if err := makeVenv(py, venv); err != nil {
|
||||||
|
t.Fatalf("makeVenv: %v", err)
|
||||||
|
}
|
||||||
|
if ok, ver := venvUsable(venv); !ok {
|
||||||
|
t.Fatalf("an environment built from the interpreter setup just chose "+
|
||||||
|
"reported itself unusable (%s)", ver)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The two states that must not be confused with a working one.
|
||||||
|
empty := filepath.Join(t.TempDir(), "gone")
|
||||||
|
if ok, _ := venvUsable(empty); ok {
|
||||||
|
t.Error("a missing environment was reported usable")
|
||||||
|
}
|
||||||
|
broken := filepath.Join(t.TempDir(), "broken")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(venvPython(broken)), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(venvPython(broken), []byte("not an interpreter"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ok, ver := venvUsable(broken); ok {
|
||||||
|
t.Errorf("a half-created environment was reported usable (%s)", ver)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -325,7 +325,7 @@ func cmdRun() error {
|
|||||||
cfg.ClientID, cfg.SiteID, cfg.BrokerURL)
|
cfg.ClientID, cfg.SiteID, cfg.BrokerURL)
|
||||||
client, err := mqtt.NewClient(mqtt.ClientOptions{
|
client, err := mqtt.NewClient(mqtt.ClientOptions{
|
||||||
BrokerURL: cfg.BrokerURL,
|
BrokerURL: cfg.BrokerURL,
|
||||||
ClientID: "behavision-" + cfg.ClientID + "-" + cfg.SiteID,
|
ClientID: cfg.MQTTClientID(),
|
||||||
Username: cfg.BrokerUsername, Password: cfg.BrokerPassword,
|
Username: cfg.BrokerUsername, Password: cfg.BrokerPassword,
|
||||||
CAFile: cfg.BrokerCAFile, Log: logger,
|
CAFile: cfg.BrokerCAFile, Log: logger,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -30,12 +30,12 @@ import (
|
|||||||
// argument, and it is why the wanted-check comes first and the push stops the
|
// argument, and it is why the wanted-check comes first and the push stops the
|
||||||
// moment the server says the last viewer has gone.
|
// moment the server says the last viewer has gone.
|
||||||
type Live struct {
|
type Live struct {
|
||||||
Engine *EngineClient
|
Engine *EngineClient
|
||||||
Cloud *CloudClient
|
Cloud *CloudClient
|
||||||
Log *log.Logger
|
Log *log.Logger
|
||||||
FPS float64
|
FPS float64
|
||||||
Width int
|
Width int
|
||||||
Quality int
|
Quality int
|
||||||
}
|
}
|
||||||
|
|
||||||
// Defaults, measured against the office camera rather than guessed.
|
// Defaults, measured against the office camera rather than guessed.
|
||||||
|
|||||||
@@ -8,7 +8,9 @@
|
|||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"crypto/rand"
|
||||||
"encoding/base64"
|
"encoding/base64"
|
||||||
|
"encoding/hex"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
@@ -83,6 +85,10 @@ type Config struct {
|
|||||||
// Queue.
|
// Queue.
|
||||||
SpoolMax int `json:"spool_max"`
|
SpoolMax int `json:"spool_max"`
|
||||||
|
|
||||||
|
// InstallID distinguishes THIS installation from every other one claimed
|
||||||
|
// to the same site. See MQTTClientID.
|
||||||
|
InstallID string `json:"install_id,omitempty"`
|
||||||
|
|
||||||
path string
|
path string
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -124,6 +130,14 @@ func Load(path string) (Config, error) {
|
|||||||
return cfg, fmt.Errorf("config %s: %w", path, err)
|
return cfg, fmt.Errorf("config %s: %w", path, err)
|
||||||
}
|
}
|
||||||
cfg.path = path
|
cfg.path = path
|
||||||
|
// Minted on first load and written back, so an installation that predates
|
||||||
|
// this field gets one without anybody doing anything. Best effort: a
|
||||||
|
// read-only config still yields a working id for this run, it is simply
|
||||||
|
// not the same one next time.
|
||||||
|
if cfg.InstallID == "" {
|
||||||
|
cfg.InstallID = newInstallID()
|
||||||
|
_ = cfg.Save(path)
|
||||||
|
}
|
||||||
for _, field := range []*string{&cfg.BrokerPassword, &cfg.APIPassword,
|
for _, field := range []*string{&cfg.BrokerPassword, &cfg.APIPassword,
|
||||||
&cfg.SessionToken, &cfg.SessionRefresh, &cfg.AgentToken} {
|
&cfg.SessionToken, &cfg.SessionRefresh, &cfg.AgentToken} {
|
||||||
plain, err := reveal(*field)
|
plain, err := reveal(*field)
|
||||||
@@ -210,3 +224,41 @@ func reveal(stored string) (string, error) {
|
|||||||
}
|
}
|
||||||
return string(plain), nil
|
return string(plain), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MQTTClientID names this INSTALLATION, not this site.
|
||||||
|
//
|
||||||
|
// It was `behavision-<client>-<site>`, which is the same string on every
|
||||||
|
// computer claimed to one shop. MQTT requires client ids to be unique and a
|
||||||
|
// broker enforces it by disconnecting the older session when a new one
|
||||||
|
// arrives with the same id - so two machines on one site take turns kicking
|
||||||
|
// each other off, forever. Measured on a second Mac claimed to a live shop:
|
||||||
|
//
|
||||||
|
// broker connected / broker connection lost: EOF / broker connected / ...
|
||||||
|
//
|
||||||
|
// The damage is not confined to the new machine. The shop's own till is the
|
||||||
|
// other half of that loop, so somebody signing in on a laptop to look at the
|
||||||
|
// product stops the shop delivering visits - and nothing at either end says
|
||||||
|
// why, because from each side it reads as an unstable network.
|
||||||
|
//
|
||||||
|
// The site stays in the id because it is what a broker log is read by, and
|
||||||
|
// the random half is short for the same reason. `CleanSession(true)` means
|
||||||
|
// there is no session state for a changed id to strand.
|
||||||
|
func (c Config) MQTTClientID() string {
|
||||||
|
id := c.InstallID
|
||||||
|
if id == "" {
|
||||||
|
// A config that could not be written still has to produce a UNIQUE
|
||||||
|
// id, or this falls straight back into the collision it exists to
|
||||||
|
// prevent. Per-run is the right failure: the connection works and the
|
||||||
|
// only cost is a new name in the broker's log after a restart.
|
||||||
|
id = newInstallID()
|
||||||
|
}
|
||||||
|
return "behavision-" + c.ClientID + "-" + c.SiteID + "-" + id
|
||||||
|
}
|
||||||
|
|
||||||
|
func newInstallID() string {
|
||||||
|
b := make([]byte, 4)
|
||||||
|
if _, err := rand.Read(b); err != nil {
|
||||||
|
return "x"
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(b)
|
||||||
|
}
|
||||||
|
|||||||
88
agent/pkg/config/installid_test.go
Normal file
88
agent/pkg/config/installid_test.go
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The bug this exists to prevent, measured on a second Mac claimed to a live
|
||||||
|
// shop: MQTT requires client ids to be unique, and a broker enforces it by
|
||||||
|
// disconnecting the older session when a new one arrives with the same id. The
|
||||||
|
// id was `behavision-<client>-<site>` - identical on every computer claimed to
|
||||||
|
// one shop - so the two took turns kicking each other off:
|
||||||
|
//
|
||||||
|
// broker connected / broker connection lost: EOF / broker connected / ...
|
||||||
|
//
|
||||||
|
// The damage is not confined to the new machine. The shop's own till is the
|
||||||
|
// other half of that loop, so somebody signing in on a laptop to look at the
|
||||||
|
// product stops the shop delivering visits.
|
||||||
|
func TestTwoInstallsOnOneSiteGetDifferentClientIDs(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
one := writeClaimed(t, filepath.Join(dir, "a.json"))
|
||||||
|
two := writeClaimed(t, filepath.Join(dir, "b.json"))
|
||||||
|
|
||||||
|
if one.MQTTClientID() == two.MQTTClientID() {
|
||||||
|
t.Fatalf("both installs answered to %q; the broker will disconnect one "+
|
||||||
|
"whenever the other connects", one.MQTTClientID())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the same install keeps its name across restarts, or a broker log is a
|
||||||
|
// list of strangers and nobody can tell one till from a stream of new ones.
|
||||||
|
func TestOneInstallKeepsItsClientIDAcrossRestarts(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "agent.json")
|
||||||
|
first := writeClaimed(t, path)
|
||||||
|
|
||||||
|
again, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("reload: %v", err)
|
||||||
|
}
|
||||||
|
if got, want := again.MQTTClientID(), first.MQTTClientID(); got != want {
|
||||||
|
t.Errorf("after a restart the id was %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The site stays in the id: it is what somebody reading a broker log is
|
||||||
|
// reading FOR, and an opaque random string would make every connection
|
||||||
|
// anonymous.
|
||||||
|
func TestTheClientIDStillNamesTheShop(t *testing.T) {
|
||||||
|
c := Config{ClientID: "tenext-retail", SiteID: "chennai", InstallID: "abcd1234"}
|
||||||
|
id := c.MQTTClientID()
|
||||||
|
for _, want := range []string{"tenext-retail", "chennai", "abcd1234"} {
|
||||||
|
if !strings.Contains(id, want) {
|
||||||
|
t.Errorf("client id %q does not contain %q", id, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A config that could not be written still has to produce a UNIQUE id, or a
|
||||||
|
// read-only install falls straight back into the collision. Per-run is the
|
||||||
|
// right failure: the connection works, and the only cost is a new name in the
|
||||||
|
// broker's log after a restart.
|
||||||
|
func TestAnUnsavedConfigStillGetsAUniqueID(t *testing.T) {
|
||||||
|
a := Config{ClientID: "c", SiteID: "s"}
|
||||||
|
b := Config{ClientID: "c", SiteID: "s"}
|
||||||
|
if a.MQTTClientID() == b.MQTTClientID() {
|
||||||
|
t.Fatal("two configs with no install id produced the same client id")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeClaimed(t *testing.T, path string) Config {
|
||||||
|
t.Helper()
|
||||||
|
cfg := Defaults()
|
||||||
|
cfg.ClientID, cfg.SiteID = "tenext-retail", "chennai"
|
||||||
|
if err := cfg.Save(path); err != nil {
|
||||||
|
t.Fatalf("save: %v", err)
|
||||||
|
}
|
||||||
|
// Loading is what mints the id, so an installation that predates the
|
||||||
|
// field gets one without anybody doing anything.
|
||||||
|
got, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load: %v", err)
|
||||||
|
}
|
||||||
|
if got.InstallID == "" {
|
||||||
|
t.Fatal("loading a config without an install id did not mint one")
|
||||||
|
}
|
||||||
|
return got
|
||||||
|
}
|
||||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
import shutil
|
import shutil
|
||||||
|
import ssl
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -35,6 +36,54 @@ _COPY_MAP = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _https_context() -> "ssl.SSLContext | None":
|
||||||
|
"""The CA store to trust, or None to use whatever Python defaults to.
|
||||||
|
|
||||||
|
Returning None first is deliberate. On Windows and on a Homebrew or
|
||||||
|
system Python, the default context reads the machine's own certificate
|
||||||
|
store - which is what makes a corporate proxy with its own root CA work.
|
||||||
|
Replacing that with certifi's bundle unconditionally would break every
|
||||||
|
site that has one, in order to fix a different platform.
|
||||||
|
|
||||||
|
The platform this fixes is a python.org macOS build. It ships its own
|
||||||
|
OpenSSL with NO trust store, and populates one only when somebody
|
||||||
|
double-clicks `Install Certificates.command` in the Python folder -
|
||||||
|
which nobody installing face-recognition software has any reason to know
|
||||||
|
about. Every HTTPS request from that interpreter fails with:
|
||||||
|
|
||||||
|
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
|
||||||
|
certificate verify failed: unable to get local issuer certificate
|
||||||
|
|
||||||
|
Measured on a colleague's Mac: the engine installed perfectly and then
|
||||||
|
could not download a 230 KB model file, ending setup in forty lines of
|
||||||
|
traceback about `_ssl.c`.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import certifi
|
||||||
|
except ImportError: # pragma: no cover - certifi ships with requests
|
||||||
|
return None
|
||||||
|
return ssl.create_default_context(cafile=certifi.where())
|
||||||
|
|
||||||
|
|
||||||
|
def _urlopen(url: str, timeout: float = 60.0):
|
||||||
|
"""Open a URL, falling back to certifi's CA bundle on a verify failure.
|
||||||
|
|
||||||
|
Default first, certifi second, so the fix is additive: a machine whose
|
||||||
|
own store works keeps using it, and one with no store at all gets a
|
||||||
|
bundle rather than a traceback. certifi is already here - `requests` is a
|
||||||
|
hard dependency and brings it.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return urllib.request.urlopen(url, timeout=timeout)
|
||||||
|
except ssl.SSLCertVerificationError:
|
||||||
|
ctx = _https_context()
|
||||||
|
if ctx is None:
|
||||||
|
raise
|
||||||
|
log.info("the system certificate store could not verify %s; "
|
||||||
|
"using the bundled CA list", url.split("/")[2])
|
||||||
|
return urllib.request.urlopen(url, timeout=timeout, context=ctx)
|
||||||
|
|
||||||
|
|
||||||
def _fetch(url: str, dest: Path, label: str) -> None:
|
def _fetch(url: str, dest: Path, label: str) -> None:
|
||||||
"""Download with progress on stdout the supervisor can read.
|
"""Download with progress on stdout the supervisor can read.
|
||||||
|
|
||||||
@@ -56,7 +105,22 @@ def _fetch(url: str, dest: Path, label: str) -> None:
|
|||||||
last = pct
|
last = pct
|
||||||
log.info("download: %s %d%%", label, pct)
|
log.info("download: %s %d%%", label, pct)
|
||||||
|
|
||||||
urllib.request.urlretrieve(url, dest, hook)
|
# Streamed rather than urlretrieve, only because urlretrieve offers no way
|
||||||
|
# to pass an SSL context and the whole point here is choosing one. The
|
||||||
|
# `download: <label> <n>%` lines are a contract: the supervisor parses
|
||||||
|
# them (`progressRe`) to put first-run progress in the tray, and without
|
||||||
|
# them a shop PC shows a stopped engine for five minutes after install.
|
||||||
|
with _urlopen(url) as resp:
|
||||||
|
total = int(resp.headers.get("Content-Length") or 0)
|
||||||
|
blocks, block_size = 0, 64 * 1024
|
||||||
|
with open(dest, "wb") as out:
|
||||||
|
while True:
|
||||||
|
chunk = resp.read(block_size)
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
out.write(chunk)
|
||||||
|
blocks += 1
|
||||||
|
hook(blocks, block_size, total)
|
||||||
log.info("download: %s 100%%", label)
|
log.info("download: %s 100%%", label)
|
||||||
|
|
||||||
|
|
||||||
@@ -91,7 +155,7 @@ def setup_models(models_dir: Path) -> "list[str]":
|
|||||||
import io
|
import io
|
||||||
import zipfile
|
import zipfile
|
||||||
|
|
||||||
with urllib.request.urlopen(BUFFALO_SC_URL) as resp:
|
with _urlopen(BUFFALO_SC_URL) as resp:
|
||||||
payload = io.BytesIO(resp.read())
|
payload = io.BytesIO(resp.read())
|
||||||
with zipfile.ZipFile(payload) as zf, \
|
with zipfile.ZipFile(payload) as zf, \
|
||||||
zf.open("w600k_mbf.onnx") as src, \
|
zf.open("w600k_mbf.onnx") as src, \
|
||||||
|
|||||||
@@ -44,6 +44,9 @@ type App struct {
|
|||||||
broker *agentmqtt.Client
|
broker *agentmqtt.Client
|
||||||
stopBridge func()
|
stopBridge func()
|
||||||
hookURL string
|
hookURL string
|
||||||
|
// The resolved engine command, so engineMissing() and the supervisor are
|
||||||
|
// never looking at two different paths.
|
||||||
|
engineExe string
|
||||||
// Relays camera feeds to the webview so the engine's credential never has
|
// Relays camera feeds to the webview so the engine's credential never has
|
||||||
// to travel in an <img> src, which a Chromium webview would strip anyway.
|
// to travel in an <img> src, which a Chromium webview would strip anyway.
|
||||||
proxy *streamProxy
|
proxy *streamProxy
|
||||||
@@ -109,6 +112,9 @@ func (a *App) startup(ctx context.Context) {
|
|||||||
if exe != "" && !filepath.IsAbs(exe) {
|
if exe != "" && !filepath.IsAbs(exe) {
|
||||||
exe = filepath.Join(agentpaths.InstallRoot(), exe)
|
exe = filepath.Join(agentpaths.InstallRoot(), exe)
|
||||||
}
|
}
|
||||||
|
a.mu.Lock()
|
||||||
|
a.engineExe = exe
|
||||||
|
a.mu.Unlock()
|
||||||
logFile, _ := agentengine.LogFile(agentpaths.EngineLog())
|
logFile, _ := agentengine.LogFile(agentpaths.EngineLog())
|
||||||
a.sup = agentengine.New(agentengine.Options{
|
a.sup = agentengine.New(agentengine.Options{
|
||||||
Command: func(c context.Context) *exec.Cmd {
|
Command: func(c context.Context) *exec.Cmd {
|
||||||
@@ -154,13 +160,55 @@ func (a *App) startup(ctx context.Context) {
|
|||||||
// not run yet, starting the supervisor would loop on a missing executable
|
// not run yet, starting the supervisor would loop on a missing executable
|
||||||
// with nothing useful to say. The Start button still exists for the one
|
// with nothing useful to say. The Start button still exists for the one
|
||||||
// case where somebody has deliberately stopped it.
|
// case where somebody has deliberately stopped it.
|
||||||
if _, err := os.Stat(exe); err == nil {
|
if why := a.engineMissing(); why == "" {
|
||||||
a.sup.Start()
|
a.sup.Start()
|
||||||
} else {
|
} else {
|
||||||
log.Printf("engine not installed yet (%s); run behavision-setup, then Start", exe)
|
log.Printf("%s (looked for %s)", why, exe)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// engineMissing says, in a sentence somebody can act on, why recognition
|
||||||
|
// cannot start here - or "" when it can.
|
||||||
|
//
|
||||||
|
// It exists because the answer was only ever given at startup, to a log file
|
||||||
|
// nobody on a shop counter opens. Pressing Start went straight to the
|
||||||
|
// supervisor, which reported what exec reported:
|
||||||
|
//
|
||||||
|
// engine failed to start: fork/exec /private/var/folders/c2/.../
|
||||||
|
// AppTranslocation/500A5354-.../d/Behavision.app/Contents/MacOS/engine/
|
||||||
|
// behavision: no such file or directory
|
||||||
|
//
|
||||||
|
// Every word of that is true and none of it says "run the setup tool". One
|
||||||
|
// function, consulted by the startup path, the Start button and the status
|
||||||
|
// panel, so the three cannot give three different accounts of one fact.
|
||||||
|
func (a *App) engineMissing() string {
|
||||||
|
a.mu.RLock()
|
||||||
|
exe := a.engineExe
|
||||||
|
a.mu.RUnlock()
|
||||||
|
if exe == "" {
|
||||||
|
return "The recognition engine is not set up on this computer yet."
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(exe); err == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
msg := "The recognition engine is not installed on this computer yet. " +
|
||||||
|
"Run behavision-setup from the folder you unzipped, then press Start."
|
||||||
|
// macOS quarantines a downloaded app it cannot verify and runs it from a
|
||||||
|
// randomly named READ-ONLY copy - App Translocation. Every relative path
|
||||||
|
// then resolves inside that copy, which is why the engine folder appears
|
||||||
|
// to be missing from a bundle that plainly contains one, and why an
|
||||||
|
// install into it would not survive a restart. Detectable, unguessable,
|
||||||
|
// and fixed by one drag; saying nothing leaves somebody re-running a
|
||||||
|
// setup tool that cannot win.
|
||||||
|
if strings.Contains(exe, "/AppTranslocation/") {
|
||||||
|
msg = "macOS is running Behavision from a temporary read-only copy, " +
|
||||||
|
"because it was opened straight from Downloads. Move Behavision " +
|
||||||
|
"to your Applications folder and open it from there, then run " +
|
||||||
|
"behavision-setup."
|
||||||
|
}
|
||||||
|
return msg
|
||||||
|
}
|
||||||
|
|
||||||
// webhookURL is the loopback address the bridge is listening on, or empty
|
// webhookURL is the loopback address the bridge is listening on, or empty
|
||||||
// before it has started.
|
// before it has started.
|
||||||
func (a *App) webhookURL() string {
|
func (a *App) webhookURL() string {
|
||||||
@@ -242,7 +290,7 @@ func (a *App) startPipeline(ctx context.Context) {
|
|||||||
}
|
}
|
||||||
client, err := agentmqtt.NewClient(agentmqtt.ClientOptions{
|
client, err := agentmqtt.NewClient(agentmqtt.ClientOptions{
|
||||||
BrokerURL: a.cfg.BrokerURL,
|
BrokerURL: a.cfg.BrokerURL,
|
||||||
ClientID: "behavision-" + a.cfg.ClientID + "-" + a.cfg.SiteID,
|
ClientID: a.cfg.MQTTClientID(),
|
||||||
Username: a.cfg.BrokerUsername, Password: a.cfg.BrokerPassword,
|
Username: a.cfg.BrokerUsername, Password: a.cfg.BrokerPassword,
|
||||||
CAFile: a.cfg.BrokerCAFile, Log: logger,
|
CAFile: a.cfg.BrokerCAFile, Log: logger,
|
||||||
})
|
})
|
||||||
@@ -595,6 +643,11 @@ func (a *App) EngineStatus() EngineStatus {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
out.Error = err.Error()
|
out.Error = err.Error()
|
||||||
}
|
}
|
||||||
|
// The supervisor's own error is an exec failure; this replaces it with
|
||||||
|
// the reason, which is the part that tells somebody what to do.
|
||||||
|
if why := a.engineMissing(); why != "" {
|
||||||
|
out.Error = why
|
||||||
|
}
|
||||||
ctx, cancel := context.WithTimeout(a.ctx, 4*time.Second)
|
ctx, cancel := context.WithTimeout(a.ctx, 4*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
// A running process is not a working engine: on a memory-starved box the
|
// A running process is not a working engine: on a memory-starved box the
|
||||||
@@ -609,6 +662,13 @@ func (a *App) EngineStatus() EngineStatus {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (a *App) StartEngine() EngineStatus {
|
func (a *App) StartEngine() EngineStatus {
|
||||||
|
// Refused rather than attempted. Handing a missing path to the supervisor
|
||||||
|
// produces a retry loop and an exec error for a message.
|
||||||
|
if why := a.engineMissing(); why != "" {
|
||||||
|
st := a.EngineStatus()
|
||||||
|
st.Error = why
|
||||||
|
return st
|
||||||
|
}
|
||||||
if a.sup != nil {
|
if a.sup != nil {
|
||||||
a.sup.Start()
|
a.sup.Start()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,8 +4,9 @@ version = "1.1.0"
|
|||||||
description = "Production face recognition over RTSP"
|
description = "Production face recognition over RTSP"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"numpy>=1.26,<2.0",
|
# See requirements.txt for why numpy is uncapped and opencv is not.
|
||||||
"opencv-python>=4.8.1",
|
"numpy>=1.26,<3.0",
|
||||||
|
"opencv-python>=4.8.1,<5",
|
||||||
"onnxruntime>=1.16",
|
"onnxruntime>=1.16",
|
||||||
"fastapi>=0.110",
|
"fastapi>=0.110",
|
||||||
"uvicorn>=0.29",
|
"uvicorn>=0.29",
|
||||||
|
|||||||
@@ -1,5 +1,31 @@
|
|||||||
numpy>=1.26,<2.0
|
# numpy 2 is allowed, and that is what lets this install on a current Python.
|
||||||
opencv-python>=4.8.1
|
# `<2.0` capped the resolver at numpy 1.26.4, whose newest wheel is cp312, so
|
||||||
|
# on a Mac with Python 3.14 pip fell back to BUILDING numpy from source and
|
||||||
|
# died in clang - two screens of C compiler output on a shop counter, for a
|
||||||
|
# version choice made silently by behavision-setup.
|
||||||
|
#
|
||||||
|
# Measured before changing it, nine runs of the detector guard per combination
|
||||||
|
# on one machine:
|
||||||
|
#
|
||||||
|
# numpy 1.26 / cv2 4.11 9 passed, 0 crashed
|
||||||
|
# numpy 2.0 / cv2 4.11 8 passed, 1 crashed
|
||||||
|
# numpy 1.26 / cv2 4.14 3 passed, 6 crashed
|
||||||
|
# numpy 2.0 / cv2 4.14 2 passed, 7 crashed
|
||||||
|
#
|
||||||
|
# numpy is not the variable there; OpenCV is. The crash was a test racing a
|
||||||
|
# shared cv2.FaceDetectorYN on purpose - undefined behaviour in C++, which 4.11
|
||||||
|
# usually turned into an exception and 4.14 usually turns into a segfault. The
|
||||||
|
# product never shares one (Engine._build_worker builds a detector per camera),
|
||||||
|
# and the test now runs that race out of process. With that fixed, the whole
|
||||||
|
# suite is 226 passed / 2 skipped on numpy 2.0.2, five runs out of five.
|
||||||
|
#
|
||||||
|
# opencv IS capped, and the two are not the same call. Everything above was
|
||||||
|
# measured on 4.x; OpenCV 5.0 is a major release this project has never run a
|
||||||
|
# real camera or an emotion model through, and an uncapped `>=4.8.1` means
|
||||||
|
# every NEW install silently gets it while every existing one keeps 4.11. Lift
|
||||||
|
# it after running a camera on 5.x, not before.
|
||||||
|
numpy>=1.26,<3.0
|
||||||
|
opencv-python>=4.8.1,<5
|
||||||
onnxruntime>=1.16
|
onnxruntime>=1.16
|
||||||
fastapi>=0.110
|
fastapi>=0.110
|
||||||
uvicorn>=0.29
|
uvicorn>=0.29
|
||||||
|
|||||||
@@ -1,7 +1,32 @@
|
|||||||
"""cv2.FaceDetectorYN caches its input size and is not thread-safe, so camera
|
"""cv2.FaceDetectorYN caches its input size and is not thread-safe, so camera
|
||||||
workers must not share one. Skipped when the model is absent, matching the
|
workers must not share one. Skipped when the model is absent, matching the
|
||||||
faiss-optional pattern in test_index.py — the suite stays runnable with no
|
faiss-optional pattern in test_index.py — the suite stays runnable with no
|
||||||
models installed."""
|
models installed.
|
||||||
|
|
||||||
|
The premise is proved in a SUBPROCESS, and that is the whole lesson of this
|
||||||
|
file. The first version raced a shared detector in-process and asserted that
|
||||||
|
an exception came back, because on OpenCV 4.11 one usually did. It is a data
|
||||||
|
race in C++: what it produces is undefined, and on 4.14 what it mostly
|
||||||
|
produces is a segmentation fault. Measured, nine runs each, same machine:
|
||||||
|
|
||||||
|
numpy 1.26 / cv2 4.11 9 passed, 0 crashed
|
||||||
|
numpy 2.0 / cv2 4.11 8 passed, 1 crashed
|
||||||
|
numpy 1.26 / cv2 4.14 3 passed, 6 crashed
|
||||||
|
numpy 2.0 / cv2 4.14 2 passed, 7 crashed
|
||||||
|
|
||||||
|
numpy is not the variable; OpenCV is, and 4.11 crashing once says the hazard
|
||||||
|
was always there and 4.11 merely survived it. A test that takes the whole
|
||||||
|
suite down two runs in three is worse than no test: it turns "we upgraded
|
||||||
|
OpenCV" into a CI failure with no failing assertion in it, which is the
|
||||||
|
hardest kind to read.
|
||||||
|
|
||||||
|
None of this reaches the product. `Engine._build_worker` constructs a
|
||||||
|
FaceDetector per camera, which is what the rule says and what the second test
|
||||||
|
here guards.
|
||||||
|
"""
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import textwrap
|
||||||
import threading
|
import threading
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -11,10 +36,16 @@ import pytest
|
|||||||
from behavision.config import Config
|
from behavision.config import Config
|
||||||
from behavision.detection import YUNET_FILENAME, FaceDetector
|
from behavision.detection import YUNET_FILENAME, FaceDetector
|
||||||
|
|
||||||
MODELS = Path(__file__).resolve().parent.parent / "models"
|
ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
MODELS = ROOT / "models"
|
||||||
pytestmark = pytest.mark.skipif(not (MODELS / YUNET_FILENAME).exists(),
|
pytestmark = pytest.mark.skipif(not (MODELS / YUNET_FILENAME).exists(),
|
||||||
reason="YuNet model not installed")
|
reason="YuNet model not installed")
|
||||||
|
|
||||||
|
# A shared detector fails probabilistically, so one clean attempt proves
|
||||||
|
# nothing. Several do: the premise holds if ANY attempt misbehaves, and only
|
||||||
|
# an unbroken run of clean ones is evidence it has stopped being true.
|
||||||
|
ATTEMPTS = 6
|
||||||
|
|
||||||
|
|
||||||
def _detector():
|
def _detector():
|
||||||
d = Config().detection
|
d = Config().detection
|
||||||
@@ -44,11 +75,37 @@ def _race(det_a, det_b):
|
|||||||
return errors
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
_SHARED_RACE = textwrap.dedent("""
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, {root!r})
|
||||||
|
from tests.test_detector_concurrency import _detector, _race
|
||||||
|
shared = _detector()
|
||||||
|
print("raced" if _race(shared, shared) else "clean")
|
||||||
|
""")
|
||||||
|
|
||||||
|
|
||||||
|
def _shared_race_outcome():
|
||||||
|
"""Run one shared-detector race out of process.
|
||||||
|
|
||||||
|
Returns "raced" (an exception came back), "crashed" (the process died,
|
||||||
|
which is the same premise arriving by a blunter route) or "clean".
|
||||||
|
"""
|
||||||
|
proc = subprocess.run([sys.executable, "-c", _SHARED_RACE.format(root=str(ROOT))],
|
||||||
|
capture_output=True, text=True, timeout=120)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
return "crashed"
|
||||||
|
return proc.stdout.strip().splitlines()[-1] if proc.stdout.strip() else "clean"
|
||||||
|
|
||||||
|
|
||||||
def test_a_shared_detector_really_does_race():
|
def test_a_shared_detector_really_does_race():
|
||||||
"""Guards the premise: if this ever stops failing, the test below is
|
"""Guards the premise: if this ever stops failing, the test below is
|
||||||
proving nothing and the per-camera split can be revisited."""
|
proving nothing and the per-camera split can be revisited."""
|
||||||
shared = _detector()
|
seen = [_shared_race_outcome() for _ in range(ATTEMPTS)]
|
||||||
assert _race(shared, shared), "expected a shared detector to race"
|
assert any(o != "clean" for o in seen), (
|
||||||
|
f"a shared detector survived {ATTEMPTS} races ({seen}) - if that is "
|
||||||
|
"reproducible, cv2.FaceDetectorYN may have become thread-safe and the "
|
||||||
|
"per-camera rule in CLAUDE.md can be revisited"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_per_camera_detectors_do_not_race():
|
def test_per_camera_detectors_do_not_race():
|
||||||
|
|||||||
105
tests/test_model_download.py
Normal file
105
tests/test_model_download.py
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
"""Downloading the models is the last step of every fresh install, and it ran
|
||||||
|
into the one macOS trap nothing else here does.
|
||||||
|
|
||||||
|
A python.org macOS build ships its own OpenSSL with NO trust store, and
|
||||||
|
populates one only when somebody double-clicks `Install Certificates.command`
|
||||||
|
in the Python folder. Measured on a colleague's Mac: the engine installed
|
||||||
|
perfectly - numpy, onnxruntime, faiss, all of it - and then could not fetch a
|
||||||
|
230 KB model file, ending setup in forty lines of traceback about `_ssl.c`.
|
||||||
|
"""
|
||||||
|
import io
|
||||||
|
import logging
|
||||||
|
import ssl
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from behavision import model_assets
|
||||||
|
|
||||||
|
|
||||||
|
class _Resp(io.BytesIO):
|
||||||
|
"""Enough of an http response for _fetch: read() and .headers."""
|
||||||
|
|
||||||
|
def __init__(self, payload: bytes):
|
||||||
|
super().__init__(payload)
|
||||||
|
self.headers = {"Content-Length": str(len(payload))}
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *exc):
|
||||||
|
self.close()
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _verify_error():
|
||||||
|
return ssl.SSLCertVerificationError(
|
||||||
|
"[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: "
|
||||||
|
"unable to get local issuer certificate")
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_machine_with_no_trust_store_falls_back_to_the_bundled_one(monkeypatch):
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def fake(url, timeout=None, context=None):
|
||||||
|
calls.append(context)
|
||||||
|
if context is None:
|
||||||
|
raise _verify_error()
|
||||||
|
return _Resp(b"ok")
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", fake)
|
||||||
|
with model_assets._urlopen("https://example.invalid/m.onnx") as resp:
|
||||||
|
assert resp.read() == b"ok"
|
||||||
|
|
||||||
|
assert len(calls) == 2, f"expected a retry, got {calls}"
|
||||||
|
# Default FIRST, and that order is the point. On Windows and on a system
|
||||||
|
# Python the default context reads the machine's own certificate store,
|
||||||
|
# which is what makes a corporate proxy with its own root CA work.
|
||||||
|
# Replacing it unconditionally would break every site that has one in
|
||||||
|
# order to fix a different platform.
|
||||||
|
assert calls[0] is None
|
||||||
|
assert isinstance(calls[1], ssl.SSLContext)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_working_trust_store_is_used_as_is(monkeypatch):
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def fake(url, timeout=None, context=None):
|
||||||
|
calls.append(context)
|
||||||
|
return _Resp(b"ok")
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", fake)
|
||||||
|
model_assets._urlopen("https://example.invalid/m.onnx").close()
|
||||||
|
assert calls == [None], "the machine's own certificate store was bypassed"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_real_network_failure_is_not_disguised_as_a_certificate_problem(monkeypatch):
|
||||||
|
def fake(url, timeout=None, context=None):
|
||||||
|
raise urllib.error.URLError("no route to host")
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", fake)
|
||||||
|
with pytest.raises(urllib.error.URLError):
|
||||||
|
model_assets._urlopen("https://example.invalid/m.onnx")
|
||||||
|
|
||||||
|
|
||||||
|
def test_progress_lines_survive_the_rewrite(tmp_path, monkeypatch, caplog):
|
||||||
|
"""`download: <label> <n>%` is a CONTRACT, not logging.
|
||||||
|
|
||||||
|
The supervisor parses it (progressRe) to put first-run progress in the
|
||||||
|
tray and the window, because the API is not up yet and a shop PC showing
|
||||||
|
a stopped engine for five minutes after install looks broken. Switching
|
||||||
|
off urlretrieve - needed because it offers no way to pass an SSL context -
|
||||||
|
is exactly the kind of change that drops it silently.
|
||||||
|
"""
|
||||||
|
payload = b"x" * (64 * 1024 * 8)
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen",
|
||||||
|
lambda url, timeout=None, context=None: _Resp(payload))
|
||||||
|
dest = tmp_path / "model.onnx"
|
||||||
|
with caplog.at_level(logging.INFO, logger="behavision.model_assets"):
|
||||||
|
model_assets._fetch("https://example.invalid/m.onnx", dest, "face detector")
|
||||||
|
|
||||||
|
assert dest.read_bytes() == payload
|
||||||
|
lines = [r.getMessage() for r in caplog.records]
|
||||||
|
pct = [ln for ln in lines if ln.startswith("download: face detector ")]
|
||||||
|
assert pct, f"no progress lines at all: {lines}"
|
||||||
|
assert "download: face detector 100%" in pct, pct
|
||||||
Reference in New Issue
Block a user