The install finished and then could not download a 230 KB file

With the version ceiling and the widened numpy pin in place, setup succeeded
on the Mac that found them - Python 3.14 chosen and accepted, numpy 2.5.3,
onnxruntime 1.30, faiss 1.15.1, the engine itself - and died on the last step,
fetching the YuNet model:

  ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
  certificate verify failed: unable to get local issuer certificate

A python.org macOS build ships its own OpenSSL with NO trust store, and
populates one only when somebody double-clicks Install Certificates.command in
the Python folder. Nobody installing face-recognition software has a reason to
know that exists, and the failure is forty lines of traceback about _ssl.c at
the end of a ten-minute install.

_urlopen tries the default context first and retries with certifi's bundle on
a verification failure. The order is the design:

- Default first, because on Windows and on a system or Homebrew Python the
  default context reads the machine's own certificate store, which is what
  makes a corporate proxy with its own root CA work. Replacing it
  unconditionally would break every site that has one to fix a different
  platform.
- certifi second, because it is already installed: requests is a hard
  dependency and brings it.
- URLError is re-raised untouched. "No route to host" and "no trust store" are
  different problems, and retrying the first with a different CA list only
  delays the real message.

urlretrieve had to go, since it offers no way to pass a context - exactly the
kind of rewrite that silently drops something. The `download: <label> <n>%`
lines are a contract: supervisor.go's progressRe parses them to put first-run
progress in the tray, because the API is not up yet and a shop PC showing a
stopped engine for five minutes looks broken. A test asserts them, and the
rewritten fetch was checked against the real URL: 232,589 bytes, sha256
identical to the model already on disk.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KGcjxF1cNLcuwc3DAPcnfj

CLAUDE.md

@@ -3559,3 +3559,43 @@ one, and why installing into it would not survive a restart. Fixed by dragging
 the app to Applications; saying nothing leaves somebody re-running a setup tool
 that cannot win. The product is unsigned, so this is the *normal* first-run
 state on every Mac, not an edge case.
+
+### And then it could not download a 230 KB file
+
+With all of the above fixed the install succeeded on that Mac - Python 3.14
+chosen and accepted, numpy 2.5.3, onnxruntime 1.30, faiss 1.15.1, the engine
+itself - and setup died on the last step, fetching the YuNet model:
+
+```
+ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
+certificate verify failed: unable to get local issuer certificate
+```
+
+A python.org macOS build ships its **own OpenSSL with no trust store**, and
+populates one only when somebody double-clicks `Install Certificates.command`
+in the Python folder. Nobody installing face-recognition software has any
+reason to know that exists, and the failure is forty lines of traceback about
+`_ssl.c` at the end of a ten-minute install.
+
+`_urlopen` tries the default context first and retries with **certifi's**
+bundle on a verification failure. The order is the whole design:
+
+- Default first, because on Windows and on a system or Homebrew Python the
+  default context reads the machine's own certificate store - which is what
+  makes a corporate proxy with its own root CA work. Replacing it
+  unconditionally would break every site that has one in order to fix a
+  different platform.
+- certifi second, because it is already installed: `requests` is a hard
+  dependency and brings it.
+- A `URLError` is re-raised untouched. "No route to host" and "no trust store"
+  are different problems, and retrying the first with a different CA list only
+  delays the real message.
+
+`urlretrieve` had to go, since it offers no way to pass a context - and that is
+exactly the kind of rewrite that silently drops something. The
+`download: <label> <n>%` lines are a **contract**: `supervisor.go`'s
+`progressRe` parses them to put first-run progress in the tray, because the API
+is not up yet and a shop PC showing a stopped engine for five minutes after
+install looks broken. `tests/test_model_download.py` asserts them, and the
+rewritten fetch was checked against the real URL: 232,589 bytes, sha256
+identical to the model already on disk.

behavision/model_assets.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import logging
 import shutil
+import ssl
 import urllib.request
 from pathlib import Path
 
@@ -35,6 +36,54 @@ _COPY_MAP = {
 }
 
 
+def _https_context() -> "ssl.SSLContext | None":
+    """The CA store to trust, or None to use whatever Python defaults to.
+
+    Returning None first is deliberate. On Windows and on a Homebrew or
+    system Python, the default context reads the machine's own certificate
+    store - which is what makes a corporate proxy with its own root CA work.
+    Replacing that with certifi's bundle unconditionally would break every
+    site that has one, in order to fix a different platform.
+
+    The platform this fixes is a python.org macOS build. It ships its own
+    OpenSSL with NO trust store, and populates one only when somebody
+    double-clicks `Install Certificates.command` in the Python folder -
+    which nobody installing face-recognition software has any reason to know
+    about. Every HTTPS request from that interpreter fails with:
+
+        ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]
+        certificate verify failed: unable to get local issuer certificate
+
+    Measured on a colleague's Mac: the engine installed perfectly and then
+    could not download a 230 KB model file, ending setup in forty lines of
+    traceback about `_ssl.c`.
+    """
+    try:
+        import certifi
+    except ImportError:  # pragma: no cover - certifi ships with requests
+        return None
+    return ssl.create_default_context(cafile=certifi.where())
+
+
+def _urlopen(url: str, timeout: float = 60.0):
+    """Open a URL, falling back to certifi's CA bundle on a verify failure.
+
+    Default first, certifi second, so the fix is additive: a machine whose
+    own store works keeps using it, and one with no store at all gets a
+    bundle rather than a traceback. certifi is already here - `requests` is a
+    hard dependency and brings it.
+    """
+    try:
+        return urllib.request.urlopen(url, timeout=timeout)
+    except ssl.SSLCertVerificationError:
+        ctx = _https_context()
+        if ctx is None:
+            raise
+        log.info("the system certificate store could not verify %s; "
+                 "using the bundled CA list", url.split("/")[2])
+        return urllib.request.urlopen(url, timeout=timeout, context=ctx)
+
+
 def _fetch(url: str, dest: Path, label: str) -> None:
     """Download with progress on stdout the supervisor can read.
 
@@ -56,7 +105,22 @@ def _fetch(url: str, dest: Path, label: str) -> None:
             last = pct
             log.info("download: %s %d%%", label, pct)
 
-    urllib.request.urlretrieve(url, dest, hook)
+    # Streamed rather than urlretrieve, only because urlretrieve offers no way
+    # to pass an SSL context and the whole point here is choosing one. The
+    # `download: <label> <n>%` lines are a contract: the supervisor parses
+    # them (`progressRe`) to put first-run progress in the tray, and without
+    # them a shop PC shows a stopped engine for five minutes after install.
+    with _urlopen(url) as resp:
+        total = int(resp.headers.get("Content-Length") or 0)
+        blocks, block_size = 0, 64 * 1024
+        with open(dest, "wb") as out:
+            while True:
+                chunk = resp.read(block_size)
+                if not chunk:
+                    break
+                out.write(chunk)
+                blocks += 1
+                hook(blocks, block_size, total)
     log.info("download: %s 100%%", label)
 
 
@@ -91,7 +155,7 @@ def setup_models(models_dir: Path) -> "list[str]":
         import io
         import zipfile
 
-        with urllib.request.urlopen(BUFFALO_SC_URL) as resp:
+        with _urlopen(BUFFALO_SC_URL) as resp:
             payload = io.BytesIO(resp.read())
         with zipfile.ZipFile(payload) as zf, \
                 zf.open("w600k_mbf.onnx") as src, \

tests/test_model_download.py

@@ -0,0 +1,105 @@
+"""Downloading the models is the last step of every fresh install, and it ran
+into the one macOS trap nothing else here does.
+
+A python.org macOS build ships its own OpenSSL with NO trust store, and
+populates one only when somebody double-clicks `Install Certificates.command`
+in the Python folder. Measured on a colleague's Mac: the engine installed
+perfectly - numpy, onnxruntime, faiss, all of it - and then could not fetch a
+230 KB model file, ending setup in forty lines of traceback about `_ssl.c`.
+"""
+import io
+import logging
+import ssl
+import urllib.request
+
+import pytest
+
+from behavision import model_assets
+
+
+class _Resp(io.BytesIO):
+    """Enough of an http response for _fetch: read() and .headers."""
+
+    def __init__(self, payload: bytes):
+        super().__init__(payload)
+        self.headers = {"Content-Length": str(len(payload))}
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        self.close()
+        return False
+
+
+def _verify_error():
+    return ssl.SSLCertVerificationError(
+        "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: "
+        "unable to get local issuer certificate")
+
+
+def test_a_machine_with_no_trust_store_falls_back_to_the_bundled_one(monkeypatch):
+    calls = []
+
+    def fake(url, timeout=None, context=None):
+        calls.append(context)
+        if context is None:
+            raise _verify_error()
+        return _Resp(b"ok")
+
+    monkeypatch.setattr(urllib.request, "urlopen", fake)
+    with model_assets._urlopen("https://example.invalid/m.onnx") as resp:
+        assert resp.read() == b"ok"
+
+    assert len(calls) == 2, f"expected a retry, got {calls}"
+    # Default FIRST, and that order is the point. On Windows and on a system
+    # Python the default context reads the machine's own certificate store,
+    # which is what makes a corporate proxy with its own root CA work.
+    # Replacing it unconditionally would break every site that has one in
+    # order to fix a different platform.
+    assert calls[0] is None
+    assert isinstance(calls[1], ssl.SSLContext)
+
+
+def test_a_working_trust_store_is_used_as_is(monkeypatch):
+    calls = []
+
+    def fake(url, timeout=None, context=None):
+        calls.append(context)
+        return _Resp(b"ok")
+
+    monkeypatch.setattr(urllib.request, "urlopen", fake)
+    model_assets._urlopen("https://example.invalid/m.onnx").close()
+    assert calls == [None], "the machine's own certificate store was bypassed"
+
+
+def test_a_real_network_failure_is_not_disguised_as_a_certificate_problem(monkeypatch):
+    def fake(url, timeout=None, context=None):
+        raise urllib.error.URLError("no route to host")
+
+    monkeypatch.setattr(urllib.request, "urlopen", fake)
+    with pytest.raises(urllib.error.URLError):
+        model_assets._urlopen("https://example.invalid/m.onnx")
+
+
+def test_progress_lines_survive_the_rewrite(tmp_path, monkeypatch, caplog):
+    """`download: <label> <n>%` is a CONTRACT, not logging.
+
+    The supervisor parses it (progressRe) to put first-run progress in the
+    tray and the window, because the API is not up yet and a shop PC showing
+    a stopped engine for five minutes after install looks broken. Switching
+    off urlretrieve - needed because it offers no way to pass an SSL context -
+    is exactly the kind of change that drops it silently.
+    """
+    payload = b"x" * (64 * 1024 * 8)
+    monkeypatch.setattr(urllib.request, "urlopen",
+                        lambda url, timeout=None, context=None: _Resp(payload))
+    dest = tmp_path / "model.onnx"
+    with caplog.at_level(logging.INFO, logger="behavision.model_assets"):
+        model_assets._fetch("https://example.invalid/m.onnx", dest, "face detector")
+
+    assert dest.read_bytes() == payload
+    lines = [r.getMessage() for r in caplog.records]
+    pct = [ln for ln in lines if ln.startswith("download: face detector ")]
+    assert pct, f"no progress lines at all: {lines}"
+    assert "download: face detector 100%" in pct, pct