Live camera view in the app, and the green light that was lying about it
Two changes, and the second was found by verifying the first. ## Watching a camera from the app, in another building Snapshots answer "is that camera working". They do not answer "what is happening in my shop right now", which is what somebody who opens the app away from the counter is asking. Head office's browser already had that answer - LiveHub plus cameras.Live, where the shop PC asks outbound whether anybody is watching and pushes JPEG frames for as long as somebody is - and the app could not reach it. cloud.CameraLive opens that feed and the app's own loopback relay re-emits it as multipart MJPEG. That is the trick: frames arrive base64 over SSE, an <img> cannot render that, and an <img> renders MJPEG natively - so a tile is an ordinary <img> pointed at loopback whether the camera is in this room or another city. - Reconnecting happens in the relay, not the page. The server caps one push at five minutes, so doing it here means the <img> never sees the stream end. - The headers are flushed before the first frame. Go writes them on the first body write, so without that the whole response waits for the shop PC to start pushing. Measured against production: 30 seconds and not even a Content-Type, which surfaces as the request timing out. - One camera at a time. Watching makes a shop PC upload, so a grid that went live at once would put an estate's worth of cameras on the wire because somebody opened a page. - live.mjpeg is behind the same per-run token as the engine routes, and a wrong token is a 404 that never reaches head office at all. - CameraLive uses its own HTTP client: the shared one's 30s timeout covers the whole response and would sever a working view every thirty seconds - the trap that made the server set WriteTimeout to zero for its own SSE endpoint. ## A camera read "Connected" for 34 minutes after the shop PC went blind Which is why the verification above looked like a failure: head office registered the viewer and no frame ever came. reportWith returns early when the engine is unreachable - correctly, it has nothing to say - so the last state it sent stays in the database looking current. Measured live: cam2 and entrance both reading Connected, in green, with last_seen_at 34 minutes old, while the heartbeat from the same PC said cameras_up 0 of 0. Two surfaces reading two stored fields and disagreeing. false could not be the answer. It means "this camera is not connecting", which sends an installer to check cabling on a camera that was working perfectly the last time anybody could ask it. So there are four states and one function: connected reported recently, and working not_connecting reported recently, and the stream will not open waiting no shop PC has ever reported this camera stale reported once, and not lately - Connected is CLEARED when stale or waiting. A stale true left in place stays available to every client reading the field directly, and leaves two fields on one object disagreeing - how the shops screen once came out labelled Working, in green, above "2 of 3 cameras not connecting". - Computed in scanCamera, so every camera anybody reads passes through it. A state computed per handler is one a handler forgets, and this had already reached three screens. - CameraStaleAfter is 5 minutes: five missed reports, not one. Same reasoning as three missed heartbeats - an indicator that cries wolf gets ignored. - An unparseable last_seen_at is stale. It should be impossible, which is why it must not fall through to the state that says everything is fine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KGcjxF1cNLcuwc3DAPcnfj
This commit is contained in:
@@ -82,6 +82,14 @@ func (a *App) startup(ctx context.Context) {
|
||||
if err := a.proxy.start(a.local.Base, a.local.User, a.local.Password); err != nil {
|
||||
log.Printf("camera relay unavailable, tiles will not load: %v", err)
|
||||
}
|
||||
// And the other direction: watching a camera in another building, through
|
||||
// head office's relay. Enabled unconditionally rather than only when a
|
||||
// session already exists, because signing in is a thing that happens
|
||||
// while the app is open - and CameraLive refuses without a session
|
||||
// anyway, so there is nothing to gate.
|
||||
if err := a.proxy.watchRemote(a.cloud.CameraLive); err != nil {
|
||||
log.Printf("remote camera view unavailable: %v", err)
|
||||
}
|
||||
|
||||
// A saved session means a shop PC that rebooted overnight comes back
|
||||
// working instead of waiting for someone to log in.
|
||||
@@ -646,6 +654,7 @@ func (a *App) Cameras() ([]map[string]any, error) {
|
||||
"id": c.ID, "camera_id": c.CameraID, "label": c.Label,
|
||||
"site": c.Site, "enabled": c.Enabled,
|
||||
"connected": c.Connected, "last_seen_at": c.LastSeenAt,
|
||||
"state": c.State, "state_note": c.StateNote,
|
||||
"snapshot": c.Snapshot, "snapshot_at": c.SnapshotAt,
|
||||
// What the screen keys off to hide Edit, Test and Check: this
|
||||
// camera is on a network this PC cannot reach.
|
||||
@@ -719,6 +728,23 @@ func (a *App) StreamURL(cameraID string) string {
|
||||
return fmt.Sprintf("http://%s/api/cameras/%s/stream.mjpeg", base, cameraID)
|
||||
}
|
||||
|
||||
// RemoteStreamURL is the live view of a camera in another building.
|
||||
//
|
||||
// The picture comes from head office's relay - the shop PC pushes frames
|
||||
// outbound because nothing can reach in - and this app re-emits them as MJPEG
|
||||
// on its own loopback, so a tile is an ordinary <img> either way. A screen
|
||||
// therefore never has to know which building it is looking at.
|
||||
//
|
||||
// Empty when the relay is not running, and the caller shows the last snapshot
|
||||
// instead. There is no useful fallback URL: the head-office endpoint needs
|
||||
// this session's bearer, which an <img> cannot send.
|
||||
func (a *App) RemoteStreamURL(cameraID string) string {
|
||||
if !a.cloud.LoggedIn() {
|
||||
return ""
|
||||
}
|
||||
return a.proxy.urlFor(cameraID, "live.mjpeg")
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------- live --
|
||||
|
||||
type LiveSnapshot struct {
|
||||
|
||||
40
desktop/frontend/dist/assets/index-6LYbNlbD.js
vendored
Normal file
40
desktop/frontend/dist/assets/index-6LYbNlbD.js
vendored
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
40
desktop/frontend/dist/assets/index-vAtlvw9l.js
vendored
40
desktop/frontend/dist/assets/index-vAtlvw9l.js
vendored
File diff suppressed because one or more lines are too long
4
desktop/frontend/dist/index.html
vendored
4
desktop/frontend/dist/index.html
vendored
@@ -4,8 +4,8 @@
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Behavision</title>
|
||||
<script type="module" crossorigin src="./assets/index-vAtlvw9l.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="./assets/index-xWw5ie4A.css">
|
||||
<script type="module" crossorigin src="./assets/index-6LYbNlbD.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="./assets/index-DOJ2bRrM.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
|
||||
@@ -38,6 +38,9 @@ export const api = {
|
||||
startPlacement: (id, seconds) => call('StartPlacementCheck', id, seconds),
|
||||
placementResult: (id) => call('PlacementResult', id),
|
||||
streamURL: (id) => call('StreamURL', id),
|
||||
// The live view of a camera in another building, relayed through head
|
||||
// office. Empty when nobody is signed in.
|
||||
remoteStreamURL: (id) => call('RemoteStreamURL', id),
|
||||
|
||||
live: () => call('Live'),
|
||||
pipelineStatus: () => call('PipelineStatus'),
|
||||
|
||||
@@ -648,3 +648,18 @@ tr.click { cursor: pointer; } tr.click:hover td { background: var(--s2); }
|
||||
}
|
||||
.viewing b { color: var(--ink); font-weight: 600; }
|
||||
.viewing svg { flex: none; margin-top: 2px; color: var(--accent); }
|
||||
|
||||
/* Watch live sits over the picture, opposite the connection pill. It is on
|
||||
the tile rather than in the button row because it is about the picture, and
|
||||
because the row it would otherwise join is hidden on a remote camera. */
|
||||
.camview .btn.watch {
|
||||
position: absolute;
|
||||
right: 10px;
|
||||
bottom: 10px;
|
||||
background: rgba(0, 0, 0, .55);
|
||||
border-color: rgba(255, 255, 255, .25);
|
||||
color: #fff;
|
||||
backdrop-filter: blur(6px);
|
||||
}
|
||||
.camview .btn.watch:hover { background: rgba(0, 0, 0, .72); }
|
||||
.camview .btn.watch.on { background: var(--accent); border-color: var(--accent); color: #fff; }
|
||||
|
||||
@@ -27,6 +27,19 @@ export default function Cameras() {
|
||||
// than one that is absent.
|
||||
const remote = cams.some(c => c.remote)
|
||||
const streams = useStreamURLs(remote ? [] : cams)
|
||||
// ONE camera at a time, and that is a cost decision rather than a layout
|
||||
// one. A remote view makes the shop computer upload frames for as long as
|
||||
// somebody is watching, so a grid that all went live at once would put an
|
||||
// estate's worth of cameras on the wire because somebody opened a page.
|
||||
const [watching, setWatching] = useState(null)
|
||||
const [watchURL, setWatchURL] = useState('')
|
||||
useEffect(() => {
|
||||
let alive = true
|
||||
if (!watching) { setWatchURL(''); return }
|
||||
api.remoteStreamURL(watching).then(u => { if (alive) setWatchURL(u || '') })
|
||||
.catch(() => { if (alive) setWatchURL('') })
|
||||
return () => { alive = false }
|
||||
}, [watching])
|
||||
|
||||
async function remove(cam) {
|
||||
if (!confirm(`Remove ${cam.id}? Recognition from it stops immediately.`)) return
|
||||
@@ -49,8 +62,9 @@ export default function Cameras() {
|
||||
|
||||
{remote && <div className="viewing">
|
||||
<b>Viewing your shops from here.</b> These cameras are wired to the shop
|
||||
computers, so they are set up and checked there. The picture is each
|
||||
camera's most recent frame, not live video.
|
||||
computers, so they are set up and checked there. Each tile shows that
|
||||
camera's most recent frame; <b>Watch live</b> asks the shop computer to
|
||||
send video for as long as you are looking.
|
||||
</div>}
|
||||
|
||||
{error && <div className="err"><Icon.Warning size={15} />{error}</div>}
|
||||
@@ -66,7 +80,10 @@ export default function Cameras() {
|
||||
</div>
|
||||
: <div className="camgrid">
|
||||
{cams.map(c => (
|
||||
<CameraCard key={c.id} cam={c} stream={streams[c.id]}
|
||||
<CameraCard key={c.id} cam={c}
|
||||
stream={watching === c.id ? watchURL : streams[c.id]}
|
||||
watching={watching === c.id}
|
||||
onWatch={() => setWatching(watching === c.id ? null : c.id)}
|
||||
onEdit={() => setEditing(c)} onCheck={() => setCheck(c.id)} onRemove={() => remove(c)} />
|
||||
))}
|
||||
</div>}
|
||||
@@ -78,14 +95,23 @@ export default function Cameras() {
|
||||
)
|
||||
}
|
||||
|
||||
function CameraCard({ cam, stream, onEdit, onCheck, onRemove }) {
|
||||
function CameraCard({ cam, stream, watching, onWatch, onEdit, onCheck, onRemove }) {
|
||||
// Three states, not two, and the third is why `connected` is a pointer on
|
||||
// the wire: null means no shop computer has reported on this camera yet,
|
||||
// which reads as waiting rather than as a fault to go and investigate.
|
||||
const conn = cam.connected === undefined || cam.connected === null
|
||||
? (cam.remote ? { tone: 'idle', label: 'Waiting for the shop computer' }
|
||||
: { tone: 'idle', label: 'Engine stopped' })
|
||||
: cam.connected ? { tone: 'ok', label: 'Connected' } : { tone: 'bad', label: 'Not connecting' }
|
||||
const conn = cam.remote
|
||||
// Four states, decided once by the server. `stale` is the one that was
|
||||
// missing: the shop computer reports nothing when it cannot reach its own
|
||||
// engine, so its last report used to sit there reading Connected -
|
||||
// measured at 34 minutes on the live estate.
|
||||
? ({ connected: { tone: 'ok', label: 'Connected' },
|
||||
not_connecting: { tone: 'bad', label: 'Not connecting' },
|
||||
stale: { tone: 'warn', label: 'Not reporting' } }[cam.state]
|
||||
|| { tone: 'idle', label: 'Waiting for the shop computer' })
|
||||
: cam.connected === undefined || cam.connected === null
|
||||
? { tone: 'idle', label: 'Engine stopped' }
|
||||
: cam.connected ? { tone: 'ok', label: 'Connected' }
|
||||
: { tone: 'bad', label: 'Not connecting' }
|
||||
// The last placement verdict, so "proven" survives closing the sheet. Only
|
||||
// `good` is a pass: marginal means half the visitors are silently discarded.
|
||||
// Never asked for a remote camera: that answer lives on the shop computer,
|
||||
@@ -105,6 +131,9 @@ function CameraCard({ cam, stream, onEdit, onCheck, onRemove }) {
|
||||
? <img src={stream || shot} alt={cam.id} />
|
||||
: <div className="placeholder"><Icon.NoCamera size={34} /></div>}
|
||||
<span className={`pill ${conn.tone === 'idle' ? '' : conn.tone} over`}><i className={`dot ${conn.tone}`} />{conn.label}</span>
|
||||
{cam.remote && <button className={`btn sm watch ${watching ? 'on' : ''}`} onClick={onWatch}>
|
||||
<Icon.Play size={13} />{watching ? 'Stop watching' : 'Watch live'}
|
||||
</button>}
|
||||
</div>
|
||||
<div className="cambody">
|
||||
<div className="camtitle">
|
||||
@@ -121,9 +150,13 @@ function CameraCard({ cam, stream, onEdit, onCheck, onRemove }) {
|
||||
</div>
|
||||
{cam.remote
|
||||
? <div className="camproof">
|
||||
<span className="note">{cam.snapshot?.available
|
||||
? 'Last picture from the shop computer.'
|
||||
: cam.snapshot?.reason || 'No picture yet from the shop computer.'}</span>
|
||||
<span className="note">{cam.state_note
|
||||
? cam.state_note
|
||||
: watching
|
||||
? 'Live from the shop computer. It uploads only while you watch.'
|
||||
: cam.snapshot?.available
|
||||
? 'Last picture from the shop computer. Watch live to see it now.'
|
||||
: cam.snapshot?.reason || 'No picture yet from the shop computer.'}</span>
|
||||
</div>
|
||||
: <div className="camproof">
|
||||
<span className={`tag ${proof.tone}`}>{proof.label}</span>
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
@@ -682,6 +683,12 @@ type RemoteCamera struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Connected *bool `json:"connected"`
|
||||
LastSeenAt string `json:"last_seen_at"`
|
||||
// State is the server's single answer - connected / not_connecting /
|
||||
// waiting / stale - and the screen renders that rather than deciding
|
||||
// again from Connected. Two places deciding one fact is how a shop came
|
||||
// out labelled Working, in green, above "2 of 3 cameras not connecting".
|
||||
State string `json:"state"`
|
||||
StateNote string `json:"state_note"`
|
||||
Snapshot Photo `json:"snapshot"`
|
||||
SnapshotAt string `json:"snapshot_at"`
|
||||
}
|
||||
@@ -746,3 +753,72 @@ func (c *Client) resolveShot(ctx context.Context, camID, at string, p Photo) Pho
|
||||
p.URL, p.Auth = uri, false
|
||||
return p
|
||||
}
|
||||
|
||||
// CameraLive opens head office's live relay for one camera and returns the
|
||||
// live SSE response for the caller to read and close.
|
||||
//
|
||||
// A response rather than frames, because the consumer is the app's own
|
||||
// loopback relay: it re-emits these frames as MJPEG so an <img> can show them,
|
||||
// and buffering the stream through a channel here would only add a place for
|
||||
// frames to queue. A stale frame is worthless - the only one worth having is
|
||||
// the newest - which is the whole reason LiveHub drops rather than queues.
|
||||
//
|
||||
// There is no client timeout on this request. A live view is endless by
|
||||
// design and any deadline would cut the picture off mid-shift; the context is
|
||||
// what ends it, when the viewer navigates away.
|
||||
func (c *Client) CameraLive(ctx context.Context, cameraID string) (*http.Response, error) {
|
||||
resp, err := c.liveOnce(ctx, cameraID)
|
||||
if errors.Is(err, errTokenExpired) {
|
||||
if rerr := c.Refresh(ctx); rerr != nil {
|
||||
return nil, rerr
|
||||
}
|
||||
resp, err = c.liveOnce(ctx, cameraID)
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
func (c *Client) liveOnce(ctx context.Context, cameraID string) (*http.Response, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
|
||||
c.Base+"/api/cameras/"+url.PathEscape(cameraID)+"/live", nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Accept", "text/event-stream")
|
||||
c.mu.RLock()
|
||||
tok := c.token
|
||||
c.mu.RUnlock()
|
||||
if tok == "" {
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+tok)
|
||||
|
||||
// c.http has a 30 s timeout, which covers the whole response and would
|
||||
// therefore sever a working live view every thirty seconds - the same
|
||||
// trap that made the server set WriteTimeout to zero for its own SSE
|
||||
// endpoint. A dedicated client, with the dial bounded instead.
|
||||
hc := &http.Client{Transport: &http.Transport{
|
||||
DialContext: (&net.Dialer{Timeout: 10 * time.Second}).DialContext,
|
||||
TLSHandshakeTimeout: 10 * time.Second,
|
||||
}}
|
||||
resp, err := hc.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("cannot reach %s: %w", c.Base, err)
|
||||
}
|
||||
if resp.StatusCode == http.StatusUnauthorized {
|
||||
var e struct {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8192))
|
||||
resp.Body.Close()
|
||||
_ = json.Unmarshal(body, &e)
|
||||
if e.Error == "token_expired" {
|
||||
return nil, errTokenExpired
|
||||
}
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
if resp.StatusCode >= 400 {
|
||||
resp.Body.Close()
|
||||
return nil, fmt.Errorf("live view: %s", resp.Status)
|
||||
}
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
@@ -21,6 +21,7 @@ package main
|
||||
// session and the bytes are fetched and handed over as an object URL.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/subtle"
|
||||
"encoding/hex"
|
||||
@@ -64,6 +65,12 @@ type streamProxy struct {
|
||||
target string // engine origin, e.g. http://127.0.0.1:8010
|
||||
user string
|
||||
pass string
|
||||
|
||||
// Opens head office's live relay for one camera. Set on a computer that
|
||||
// is signed in, whether or not an engine runs here - which is the whole
|
||||
// point: watching a camera in another building is precisely the case
|
||||
// where there is no engine on this machine to ask.
|
||||
live func(ctx context.Context, cameraID string) (*http.Response, error)
|
||||
}
|
||||
|
||||
func newStreamProxy() *streamProxy { return &streamProxy{} }
|
||||
@@ -71,18 +78,49 @@ func newStreamProxy() *streamProxy { return &streamProxy{} }
|
||||
// start binds a loopback listener and begins relaying. Calling it again while
|
||||
// running is a no-op, so a restarted engine cannot leave two listeners behind.
|
||||
func (p *streamProxy) start(base, user, pass string) error {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
if p.srv != nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(base, "http://") && !strings.HasPrefix(base, "https://") {
|
||||
base = "http://" + base
|
||||
}
|
||||
if _, err := url.Parse(base); err != nil {
|
||||
return fmt.Errorf("engine base %q: %w", base, err)
|
||||
}
|
||||
if err := p.bind(); err != nil {
|
||||
return err
|
||||
}
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
p.target = strings.TrimRight(base, "/")
|
||||
p.user, p.pass = user, pass
|
||||
return nil
|
||||
}
|
||||
|
||||
// watchRemote makes the relay able to serve head office's live view, and
|
||||
// binds it if nothing else has.
|
||||
//
|
||||
// Separate from start() because the two are independent: a shop PC has both
|
||||
// an engine and a session, an owner's laptop has only a session, and a PC
|
||||
// still being set up has only an engine. Folding them together would mean a
|
||||
// computer with no engine could not watch a camera at all - which is the one
|
||||
// computer most likely to be trying to.
|
||||
func (p *streamProxy) watchRemote(fn func(context.Context, string) (*http.Response, error)) error {
|
||||
if err := p.bind(); err != nil {
|
||||
return err
|
||||
}
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
p.live = fn
|
||||
return nil
|
||||
}
|
||||
|
||||
// bind starts the loopback listener once. Calling it again while running is a
|
||||
// no-op, so neither a restarted engine nor a second sign-in can leave two
|
||||
// listeners behind.
|
||||
func (p *streamProxy) bind() error {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
if p.srv != nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
// The engine's own credential exists precisely so that the live face feed
|
||||
// is never served open - CLAUDE.md is explicit that an unauthenticated
|
||||
@@ -105,8 +143,6 @@ func (p *streamProxy) start(base, user, pass string) error {
|
||||
|
||||
p.ln = ln
|
||||
p.token = hex.EncodeToString(raw)
|
||||
p.target = strings.TrimRight(base, "/")
|
||||
p.user, p.pass = user, pass
|
||||
// No client timeout: an MJPEG stream is endless by design and any deadline
|
||||
// would cut the picture off mid-shift. The request context ends it when
|
||||
// the webview navigates away or the tile is replaced.
|
||||
@@ -130,7 +166,7 @@ func (p *streamProxy) start(base, user, pass string) error {
|
||||
func (p *streamProxy) stop() {
|
||||
p.mu.Lock()
|
||||
srv, ln := p.srv, p.ln
|
||||
p.srv, p.ln, p.token = nil, nil, ""
|
||||
p.srv, p.ln, p.token, p.live = nil, nil, "", nil
|
||||
p.mu.Unlock()
|
||||
if srv != nil {
|
||||
_ = srv.Close()
|
||||
@@ -155,6 +191,7 @@ func (p *streamProxy) urlFor(cameraID, file string) string {
|
||||
func (p *streamProxy) handle(w http.ResponseWriter, r *http.Request) {
|
||||
p.mu.RLock()
|
||||
token, target, user, pass, client := p.token, p.target, p.user, p.pass, p.client
|
||||
liveFn := p.live
|
||||
p.mu.RUnlock()
|
||||
if token == "" || client == nil {
|
||||
http.NotFound(w, r)
|
||||
@@ -190,6 +227,17 @@ func (p *streamProxy) handle(w http.ResponseWriter, r *http.Request) {
|
||||
// calls; it is here so that adding a still later is a change to a screen
|
||||
// rather than a change to the one file where a mistake is a credentialed
|
||||
// proxy onto the biometric API.
|
||||
// Head office's relay, not the engine. The two are different machines and
|
||||
// different credentials, so this returns rather than falling through.
|
||||
if parts[3] == "live.mjpeg" {
|
||||
if liveFn == nil {
|
||||
http.Error(w, "not signed in to head office", http.StatusBadGateway)
|
||||
return
|
||||
}
|
||||
p.relayRemote(w, r, cameraID, liveFn)
|
||||
return
|
||||
}
|
||||
|
||||
var enginePath string
|
||||
switch parts[3] {
|
||||
case "stream.mjpeg":
|
||||
|
||||
149
desktop/stream_remote.go
Normal file
149
desktop/stream_remote.go
Normal file
@@ -0,0 +1,149 @@
|
||||
package main
|
||||
|
||||
// Watching a camera in another building, from the app.
|
||||
//
|
||||
// The shop PC sits behind a router with no inbound route, so nothing here can
|
||||
// pull its MJPEG stream - that stream is served on the shop PC's own loopback
|
||||
// and always will be. Head office's LiveHub is the way round it: the agent
|
||||
// asks outbound whether anyone is watching and pushes JPEG frames up for
|
||||
// exactly as long as somebody is. The head-office web app already consumes
|
||||
// that; this is the same feed, for the app.
|
||||
//
|
||||
// It arrives as base64 frames over SSE, which an <img> cannot render, so this
|
||||
// re-emits them as multipart MJPEG - which an <img> renders natively, through
|
||||
// the relay that already exists for the local engine. That is what keeps ONE
|
||||
// code path in the screens: a tile points at a loopback URL and does not know
|
||||
// or care which building the picture came from.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The boundary is ours to choose; it only has to be a string the JPEG bytes
|
||||
// cannot contain, and a marker line never appears inside JPEG data.
|
||||
const mjpegBoundary = "behavisionframe"
|
||||
|
||||
// A frame is base64, so ~1.33 bytes on the wire per byte of picture. The
|
||||
// engine re-encodes to 640 px for the relay and those measure ~20 KB, so this
|
||||
// is roughly a hundredfold headroom - large enough never to clip a real frame
|
||||
// and small enough that a broken or hostile stream cannot grow this process's
|
||||
// memory without bound.
|
||||
const maxFrameLine = 8 << 20
|
||||
|
||||
func (p *streamProxy) relayRemote(w http.ResponseWriter, r *http.Request,
|
||||
cameraID string, open func(context.Context, string) (*http.Response, error)) {
|
||||
|
||||
w.Header().Set("Content-Type", "multipart/x-mixed-replace; boundary="+mjpegBoundary)
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
flusher, _ := w.(http.Flusher)
|
||||
|
||||
// Send the headers NOW, before any frame exists. Go writes them on the
|
||||
// first body write, so without this the whole response - status line
|
||||
// included - waits for the shop computer to start pushing, and a viewer
|
||||
// whose camera is slow to answer sees the REQUEST time out rather than a
|
||||
// stream that has not painted yet. Measured against production: 30
|
||||
// seconds and not even a Content-Type.
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
|
||||
// Reconnecting is normal, not an error. The server caps one push at five
|
||||
// minutes so that a tab left open for a week cannot leave a shop
|
||||
// uploading for a week - so a viewer who IS still there simply asks
|
||||
// again. Doing it here rather than in the page is what lets the <img>
|
||||
// survive the cap: it never sees the stream end.
|
||||
sent := 0
|
||||
for {
|
||||
if r.Context().Err() != nil {
|
||||
return
|
||||
}
|
||||
n, err := p.pumpRemote(w, flusher, r.Context(), cameraID, open)
|
||||
sent += n
|
||||
if r.Context().Err() != nil {
|
||||
return
|
||||
}
|
||||
// Nothing was written and the attempt failed. Writing an error body
|
||||
// now would be writing it into a multipart stream the <img> is
|
||||
// already parsing, so the picture simply stays on whatever it last
|
||||
// showed and the screen's own "not connecting" state is the report.
|
||||
if err != nil && sent == 0 {
|
||||
return
|
||||
}
|
||||
select {
|
||||
case <-r.Context().Done():
|
||||
return
|
||||
case <-time.After(1500 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// pumpRemote runs one SSE connection to exhaustion and returns how many
|
||||
// frames it forwarded.
|
||||
func (p *streamProxy) pumpRemote(w http.ResponseWriter, flusher http.Flusher,
|
||||
ctx context.Context, cameraID string,
|
||||
open func(context.Context, string) (*http.Response, error)) (int, error) {
|
||||
|
||||
resp, err := open(ctx, cameraID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
sc := bufio.NewScanner(resp.Body)
|
||||
sc.Buffer(make([]byte, 0, 64*1024), maxFrameLine)
|
||||
|
||||
var event, data string
|
||||
frames := 0
|
||||
for sc.Scan() {
|
||||
line := sc.Text()
|
||||
switch {
|
||||
case strings.HasPrefix(line, "event: "):
|
||||
event = strings.TrimSpace(line[7:])
|
||||
case strings.HasPrefix(line, "data: "):
|
||||
data = line[6:]
|
||||
case line == "":
|
||||
// End of one SSE event. `waiting` means head office has us
|
||||
// registered and the shop PC has not started pushing yet - a real
|
||||
// second or two while the agent is asked, and nothing to draw.
|
||||
if event == "frame" && data != "" {
|
||||
if err := writeMJPEGFrame(w, flusher, data); err != nil {
|
||||
return frames, err // the webview went away
|
||||
}
|
||||
frames++
|
||||
}
|
||||
event, data = "", ""
|
||||
}
|
||||
}
|
||||
return frames, sc.Err()
|
||||
}
|
||||
|
||||
func writeMJPEGFrame(w http.ResponseWriter, flusher http.Flusher, b64 string) error {
|
||||
jpg, err := base64.StdEncoding.DecodeString(b64)
|
||||
if err != nil || len(jpg) == 0 {
|
||||
// One malformed frame is not a reason to tear down a working view.
|
||||
return nil
|
||||
}
|
||||
if _, err := fmt.Fprintf(w,
|
||||
"--%s\r\nContent-Type: image/jpeg\r\nContent-Length: %d\r\n\r\n",
|
||||
mjpegBoundary, len(jpg)); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := w.Write(jpg); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := w.Write([]byte("\r\n")); err != nil {
|
||||
return err
|
||||
}
|
||||
// Flushed per frame. Anything held waiting for a full buffer is a tile
|
||||
// that stays blank, which is indistinguishable from the view not working.
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
103
desktop/stream_remote_live_test.go
Normal file
103
desktop/stream_remote_live_test.go
Normal file
@@ -0,0 +1,103 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"net/http"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/loyaly/behavision-desktop/internal/cloud"
|
||||
)
|
||||
|
||||
// The whole chain against the real head office and a real shop computer:
|
||||
//
|
||||
// TEST_CLOUD_EMAIL=... TEST_CLOUD_PASSWORD=... \
|
||||
// go test ./desktop/ -run RemoteLive -v
|
||||
//
|
||||
// Everything in stream_remote_test.go proves the relay against a fake that
|
||||
// agrees with me. Only this proves the part that cannot be faked: that a shop
|
||||
// computer behind a router with no inbound route actually pushes frames when
|
||||
// asked, that they survive base64 and SSE, and that what comes out of the
|
||||
// loopback relay is a multipart stream an <img> will paint.
|
||||
//
|
||||
// It also costs something to run, which is why it is opt-in: watching makes
|
||||
// the shop computer upload for as long as the test reads.
|
||||
func TestRemoteLiveFromProduction(t *testing.T) {
|
||||
email, pass := os.Getenv("TEST_CLOUD_EMAIL"), os.Getenv("TEST_CLOUD_PASSWORD")
|
||||
if email == "" || pass == "" {
|
||||
t.Skip("set TEST_CLOUD_EMAIL and TEST_CLOUD_PASSWORD to run against production")
|
||||
}
|
||||
base := os.Getenv("TEST_CLOUD_URL")
|
||||
if base == "" {
|
||||
base = "https://mcp.loyaly.ai"
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
c := cloud.New(base)
|
||||
if _, err := c.Login(ctx, email, pass); err != nil {
|
||||
t.Fatalf("login: %v", err)
|
||||
}
|
||||
|
||||
cams, err := c.RemoteCameras(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("cameras: %v", err)
|
||||
}
|
||||
t.Logf("%d cameras", len(cams))
|
||||
target := os.Getenv("TEST_CLOUD_CAMERA")
|
||||
for _, cam := range cams {
|
||||
conn := "waiting"
|
||||
if cam.Connected != nil {
|
||||
conn = map[bool]string{true: "connected", false: "not connecting"}[*cam.Connected]
|
||||
}
|
||||
t.Logf(" %-10s %-16s %-15s snapshot=%v", cam.CameraID, cam.Site, conn, cam.Snapshot.Available)
|
||||
if target == "" && cam.Connected != nil && *cam.Connected {
|
||||
target = cam.CameraID
|
||||
}
|
||||
}
|
||||
if target == "" {
|
||||
t.Skip("no connected camera to watch")
|
||||
}
|
||||
|
||||
p := newStreamProxy()
|
||||
if err := p.watchRemote(c.CameraLive); err != nil {
|
||||
t.Fatalf("watchRemote: %v", err)
|
||||
}
|
||||
defer p.stop()
|
||||
|
||||
rctx, rcancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
defer rcancel()
|
||||
req, _ := http.NewRequestWithContext(rctx, http.MethodGet, p.urlFor(target, "live.mjpeg"), nil)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatalf("GET relay: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
start := time.Now()
|
||||
acc, buf, frames := make([]byte, 0, 1<<20), make([]byte, 32*1024), 0
|
||||
for frames < 10 {
|
||||
n, rerr := resp.Body.Read(buf)
|
||||
acc = append(acc, buf[:n]...)
|
||||
frames = bytes.Count(acc, []byte("--"+mjpegBoundary))
|
||||
if rerr != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
el := time.Since(start)
|
||||
t.Logf("watching %q: %d frames, %d bytes, %.1fs (%.1f fps, %.0f KB/s)",
|
||||
target, frames, len(acc), el.Seconds(),
|
||||
float64(frames)/el.Seconds(), float64(len(acc))/el.Seconds()/1024)
|
||||
|
||||
if frames < 3 {
|
||||
t.Fatalf("got %d frames from a connected camera - the shop computer is "+
|
||||
"not answering head office's request to push", frames)
|
||||
}
|
||||
// Bytes that are actually a picture, not a framing header that happens to
|
||||
// be well formed. A JPEG begins FFD8.
|
||||
if !bytes.Contains(acc, []byte{0xFF, 0xD8, 0xFF}) {
|
||||
t.Error("no JPEG start marker anywhere in the stream")
|
||||
}
|
||||
}
|
||||
209
desktop/stream_remote_test.go
Normal file
209
desktop/stream_remote_test.go
Normal file
@@ -0,0 +1,209 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// jpg is a byte sequence that is not valid JPEG and does not need to be: what
|
||||
// is under test is that the bytes arrive intact and framed, not that a decoder
|
||||
// likes them.
|
||||
var jpg = []byte{0xFF, 0xD8, 'h', 'e', 'l', 'l', 'o', 0xFF, 0xD9}
|
||||
|
||||
// sseServer answers head office's live endpoint with `pushes` frames and then
|
||||
// ends the response, which is what the server's five-minute cap does.
|
||||
func sseServer(t *testing.T, frames int, hits *int32) *httptest.Server {
|
||||
t.Helper()
|
||||
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
atomic.AddInt32(hits, 1)
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
fl, _ := w.(http.Flusher)
|
||||
// Registered, nothing being pushed yet. Nothing may be drawn for it.
|
||||
fmt.Fprint(w, "event: waiting\ndata: \n\n")
|
||||
if fl != nil {
|
||||
fl.Flush()
|
||||
}
|
||||
for i := 0; i < frames; i++ {
|
||||
fmt.Fprintf(w, "event: frame\ndata: %s\n\n",
|
||||
base64.StdEncoding.EncodeToString(jpg))
|
||||
if fl != nil {
|
||||
fl.Flush()
|
||||
}
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
func openerFor(srv *httptest.Server) func(context.Context, string) (*http.Response, error) {
|
||||
return func(ctx context.Context, cam string) (*http.Response, error) {
|
||||
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+"/live/"+cam, nil)
|
||||
return http.DefaultClient.Do(req)
|
||||
}
|
||||
}
|
||||
|
||||
// The whole point: base64 frames over SSE are not something an <img> can show,
|
||||
// and a multipart MJPEG stream is. Without this the app could only ever show a
|
||||
// still, on exactly the computers that cannot reach the camera any other way.
|
||||
func TestRemoteFramesReachTheWebviewAsMJPEG(t *testing.T) {
|
||||
var hits int32
|
||||
srv := sseServer(t, 3, &hits)
|
||||
defer srv.Close()
|
||||
|
||||
p := newStreamProxy()
|
||||
if err := p.watchRemote(openerFor(srv)); err != nil {
|
||||
t.Fatalf("watchRemote: %v", err)
|
||||
}
|
||||
defer p.stop()
|
||||
|
||||
u := p.urlFor("cam2", "live.mjpeg")
|
||||
if u == "" {
|
||||
t.Fatal("no relay url; the proxy did not bind")
|
||||
}
|
||||
|
||||
// The relay reconnects for as long as the viewer is there, so the read is
|
||||
// bounded by us rather than by the stream ending - exactly as an <img>
|
||||
// would behave.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatalf("GET relay: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if ct := resp.Header.Get("Content-Type"); !strings.HasPrefix(ct, "multipart/x-mixed-replace") {
|
||||
t.Fatalf("Content-Type = %q, an <img> will not treat that as a stream", ct)
|
||||
}
|
||||
|
||||
// Read the first three frames' worth and stop; the relay would otherwise
|
||||
// go on reconnecting forever, which is the behaviour being relied on.
|
||||
want := append([]byte(fmt.Sprintf("--%s\r\nContent-Type: image/jpeg\r\nContent-Length: %d\r\n\r\n",
|
||||
mjpegBoundary, len(jpg))), jpg...)
|
||||
got := make([]byte, 0, 4096)
|
||||
buf := make([]byte, 512)
|
||||
for len(got) < 3*len(want) {
|
||||
n, rerr := resp.Body.Read(buf)
|
||||
got = append(got, buf[:n]...)
|
||||
if rerr != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
if n := bytes.Count(got, []byte("--"+mjpegBoundary)); n < 3 {
|
||||
t.Fatalf("got %d frames in %d bytes, want at least 3", n, len(got))
|
||||
}
|
||||
if !bytes.Contains(got, want) {
|
||||
t.Errorf("a frame was not framed as expected:\n%q", got[:min(len(got), 300)])
|
||||
}
|
||||
// `waiting` is a real state - head office has us registered and the shop
|
||||
// computer has not started pushing - and there is nothing to draw for it.
|
||||
// Emitting an empty part would blank a tile that already had a picture.
|
||||
if bytes.Contains(got, []byte("Content-Length: 0")) {
|
||||
t.Error("an empty frame was written for a waiting event")
|
||||
}
|
||||
}
|
||||
|
||||
// The server caps one push at five minutes so a tab left open for a week
|
||||
// cannot leave a shop uploading for a week. Reconnecting is therefore a normal
|
||||
// event, and doing it here rather than in the page is what lets the <img>
|
||||
// survive the cap - it never sees the stream end.
|
||||
func TestTheRelayReconnectsWhenHeadOfficeEndsAPush(t *testing.T) {
|
||||
var hits int32
|
||||
srv := sseServer(t, 1, &hits)
|
||||
defer srv.Close()
|
||||
|
||||
p := newStreamProxy()
|
||||
if err := p.watchRemote(openerFor(srv)); err != nil {
|
||||
t.Fatalf("watchRemote: %v", err)
|
||||
}
|
||||
defer p.stop()
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
|
||||
defer cancel()
|
||||
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, p.urlFor("cam2", "live.mjpeg"), nil)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatalf("GET relay: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Two frames means two pushes, because each push carries exactly one.
|
||||
seen, buf := 0, make([]byte, 256)
|
||||
acc := make([]byte, 0, 2048)
|
||||
for seen < 2 {
|
||||
n, rerr := resp.Body.Read(buf)
|
||||
acc = append(acc, buf[:n]...)
|
||||
seen = bytes.Count(acc, []byte("--"+mjpegBoundary))
|
||||
if rerr != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
if seen < 2 {
|
||||
t.Fatalf("got %d frames across reconnects, want 2", seen)
|
||||
}
|
||||
if got := atomic.LoadInt32(&hits); got < 2 {
|
||||
t.Errorf("head office was asked %d times, want at least 2", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Signed out, the relay must not pretend. There is no fallback URL to offer
|
||||
// either: the head-office endpoint needs this session's bearer, which an <img>
|
||||
// cannot send - so a tile that silently failed would be the only alternative.
|
||||
func TestTheRelayRefusesWhenNobodyIsSignedIn(t *testing.T) {
|
||||
p := newStreamProxy()
|
||||
if err := p.watchRemote(nil); err != nil {
|
||||
t.Fatalf("watchRemote: %v", err)
|
||||
}
|
||||
defer p.stop()
|
||||
|
||||
resp, err := http.Get(p.urlFor("cam2", "live.mjpeg"))
|
||||
if err != nil {
|
||||
t.Fatalf("GET relay: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
if resp.StatusCode != http.StatusBadGateway {
|
||||
t.Errorf("status = %d, want 502", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
// The relay is credentialed - it is a path to a live view of a shop floor -
|
||||
// and the token is the only thing standing between another local process and
|
||||
// it. live.mjpeg must be behind exactly the same door as the engine routes.
|
||||
func TestTheRemoteRouteIsBehindTheSameToken(t *testing.T) {
|
||||
var hits int32
|
||||
srv := sseServer(t, 1, &hits)
|
||||
defer srv.Close()
|
||||
|
||||
p := newStreamProxy()
|
||||
if err := p.watchRemote(openerFor(srv)); err != nil {
|
||||
t.Fatalf("watchRemote: %v", err)
|
||||
}
|
||||
defer p.stop()
|
||||
|
||||
// The right shape, the wrong value.
|
||||
parts := strings.Split(p.urlFor("cam2", "live.mjpeg"), "/")
|
||||
parts[4] = strings.Repeat("0", len(parts[4]))
|
||||
bad := strings.Join(parts, "/")
|
||||
|
||||
resp, err := http.Get(bad)
|
||||
if err != nil {
|
||||
t.Fatalf("GET relay: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
if resp.StatusCode != http.StatusNotFound {
|
||||
t.Errorf("status = %d, want 404 - and 404 rather than 403, because there is nothing here to tell an unwelcome caller they found the right door", resp.StatusCode)
|
||||
}
|
||||
if atomic.LoadInt32(&hits) != 0 {
|
||||
t.Error("a request with the wrong token still made the shop computer upload")
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user