Independent product concept.

Architecture

How an answer is produced, and what stops a bad one.

The technical view: the path one question takes through the running product, the controls around the paid model, the ontology the model is held to, and the on-site design it grows into.

In the product today

One question, six steps

  1. 1Stills

    Ten stills are cut from the clip in the browser. Faces are masked on the device before anything is sent; the video itself never leaves.

  2. 2Guard

    The server accepts only same-origin requests, ten JPEG stills of one clip under size limits, and sample stills byte-identical to the committed ones.

  3. 3Instant path

    A recorded answer, or the clip's event record, answers without any model call.

  4. 4Live model

    Otherwise one vision-model call, inside a site-wide daily budget, routed to zero-retention endpoints, with a hard timeout and bounded output.

  5. 5Gate

    Deterministic checks decide the verdict: cleared, held, not actionable, abstained or unverified.

  6. 6Question log

    Every answer shown is appended to a hash-chained log with its source and verdict.

Vision model on this deployment: openai/gpt-4o, replaceable per deployment without code changes. Current prompts: answers ask-v3-2026-09-26, event records extract-v1-2026-09-26. Every recording states the prompt that produced it; the answers on the Evidence tab came from the earlier ask-v2.

Controls

What surrounds the paid model

A public page in front of a vision model is a cost and privacy surface. These are the controls, and which of them run today.

  • Spend ceilingEvery live call is counted against a durable daily budget per visitor, per product and site-wide; when it is spent, the product answers from its recorded evidence and says so.Running
  • Request guardSame-origin only; at most ten JPEG stills of one clip, each under 300 KB and 1,280 px; questions of 300 characters or fewer.Running
  • Sample integrityStills claimed to come from a sample clip must match its committed SHA-256 hashes, so labels and duration are the server's, never the client's.Running
  • Data routingModel calls are routed to zero-data-retention endpoints first, and never to endpoints that train on or collect the data.Running
  • Error hygieneProvider errors are logged on the server only; the page receives a fixed, plain message.Running
  • Face maskingAn on-device face detector masks faces before any upload still is sent; if it cannot load, nothing is sent without the visitor's consent.Running
  • Prompt scopeThe model is told to ignore instructions inside the question or the frames; answers render as text, never as markup.Running
  • Edge processingRaw video stays on site; only masked events and pointers leave.Design
  • Role-based access and SSOQuestions and playback scoped by role; works-council viewers see aggregates only.Design
  • Anchored audit logThe question log's chain head anchored in write-once storage, so recomputing hashes is detectable.Design

Event ontology

What counts, and what must not

Nine typed events, each with an observable cue and a guard against the confusion vision models make most. The same definitions go into the model's instructions and into the event record the gate checks.

Hard-hat compliance

SafetyHigh

Counts: A person in an active work area without a rigid-shell protective helmet. A hard-hat is a rigid shell with a brim, usually a solid bright colour.

Does not count: A baseball cap, beanie, hood, or bare head styled like headwear is NOT a hard-hat - the single most common VLM confusion. If the shell/brim cannot be resolved, report 'unsure', never 'yes'.

Hi-vis compliance

SafetyMedium

Counts: A person in a vehicle/equipment area without a high-visibility garment (fluorescent yellow/orange vest or jacket, usually with reflective strips).

Does not count: A plain bright-coloured shirt without reflective strips is a weak cue - report 'unsure' rather than counting it as hi-vis.

LOTO breach (lockout/tagout)

SafetyCritical

Counts: A person with body parts inside a machine's guarded envelope or service area while no lockout device (padlock/hasp/tag on the isolation point) is visible.

Does not count: Standing NEAR a stopped machine is not a breach. Energization state usually cannot be read from a still frame - if no isolation point or lock is visible, say the frames cannot support a LOTO judgement rather than inventing one.

Forklift-pedestrian proximity

SafetyCritical

Counts: A powered industrial truck (forklift, reach truck) moving within roughly one vehicle-length of a person on foot in a shared aisle, with no physical separation.

Does not count: A parked or unmanned forklift is NOT a proximity event. A hand pallet jack is not a powered truck. If no forklift is visible in any frame, the grounded answer is 'none' - not a guess.

Restricted-zone entry

SafetyHigh

Counts: A person crossing visible floor markings, barriers, or gates into a marked exclusion or machine-guarded area.

Does not count: Only count when the marking/barrier itself is visible in frame. Faded or ambiguous markings -> 'unsure'. Do not infer zones that are not visible.

Spill / walkway obstruction

SafetyMedium

Counts: Liquid, debris, or stored material blocking a pedestrian walkway or emergency route.

Does not count: Normal work-in-progress material at a workstation is not an obstruction; the cue is material IN a walkway/aisle.

Micro-stoppage

OperationsMedium

Counts: A station that is inactive in some frames while adjacent stations remain active - visible as an operator waiting, a machine stopped mid-cycle, or material starved/blocked. Requires evidence across at least TWO frames.

Does not count: A single still frame cannot prove a stoppage (it may be mid-cycle dwell). One-frame evidence -> 'unsure, needs temporal confirmation'.

Idle station / line

OperationsMedium

Counts: A workstation or machine unmanned or inactive across consecutive frames while the area is otherwise in an active shift.

Does not count: An empty area with no evidence of an active shift is just an empty area - do not label it idle time.

Cycle complete

OperationsInfo

Counts: A unit of work visibly completed - a piece leaves a station, a weld/stitch/assembly step finishes, a package is sealed.

Does not count: Motion alone is not a completed cycle; the cue is a unit visibly finishing or leaving the station.

Production design

Extract events once on site, query them many times

Not running in this web product. The design a plant deployment grows into: perception and masking at the edge, so only events and pointers leave the site.

On site: raw video never leavesQuery time: only events and pointers
  1. 01On site

    Ingest

    Pull camera streams over RTSP/ONVIF into an on-site edge node.

  2. 02On site

    Mask and perceive

    Blur faces and bodies, then turn video into typed, timestamped events under the ontology.

  3. 03On site

    Event record

    Store events with pointers to encrypted local clips. Read once; query many times.

  4. 04Query time

    Classify the question

    Map a question to typed events, zones, shifts and time ranges.

  5. 05Query time

    Answer

    Answer from the event record; call a vision model only when the record cannot answer.

  6. 06Query time

    Gate and log

    Grade every answer, hold what fails, log every question and view.

The gate and the question log wrap every answer, whether it came from the event record or a model call.

Requirements

Must-haves, and where each one stands

P0.1Camera ingestionDesign

Events extracted continuously from RTSP/ONVIF streams without footage leaving the site.

P0.2Typed event extractionRunning

Every ontology event judged observed, not observed or cannot judge, each tied to the stills that support it.

P0.3Grounded answersRunning

Answers cite stills; 'cannot tell' when the footage cannot support an answer; never an invented event.

P0.4Citation playbackRunning

Every cited timestamp plays the footage at that moment.

P0.5Worker anonymisationPartly running

Faces masked before perception; no biometric identity stored. Running in the browser; edge masking is design-stage.

P0.6Access control and audit logPartly running

Every question logged in a hash chain. Role-based access is design-stage.

P0.7Evidence gateRunning

Any answer that contradicts a label the question depends on is held, whatever its score.

Pilot targets (hypotheses to test, not results)

  • Investigation time down 80%: from re-watching footage to reviewing a cited, gated answer.
  • Zero actions taken on held answers: every held answer routed to a person before anything happens.
  • Per-event precision published per site: measured on the site's own labelled clips before go-live.
  • Works-council and DPO sign-off with no raw-video egress: on every EU deployment.

Deliberately out of scope

  • Safety-rated machine control or e-stop: a different reliability class and liability regime.
  • Individual worker scoring: off-mission, and high-risk under the EU AI Act.
  • Selling cameras or hardware: margins and sales cycles both suffer.
  • Line-speed defect inspection: a crowded market with a different latency profile.

Build or buy

Buy perception, build judgement

  • Perception modelsBuyImproving fast and commoditising; keep the model layer swappable.
  • Event ontology and guardsBuildThe domain judgement that decides what counts.
  • Evidence gate and evaluation setBuildThe product: what makes an answer defensible.
  • Edge runtime and maskingBuild on standard partsRegulatory necessity; commodity detectors, owned orchestration.
  • Event store and queryBuy infra, build glueStandard storage; the question-to-event mapping is ours.
  • EHS and MES integrationsBuildWhere answers become corrective actions; drives retention.

Status

What runs today, and what is design

Running in the product today · 6

  • Live vision-model answers

    Custom questions and uploads get one fresh call to the configured vision model inside a daily budget. The answering model, latency, tokens and billed cost come back with every live answer.

  • Recorded answers

    Suggested questions on the sample clips replay unedited answers recorded from the live model: instant, free, and labelled with the model and date.

  • Masking before sending

    Upload stills are masked by an on-device face detector before they leave the browser; the sample stills were masked once, offline, before any answer was recorded.

  • Read-once event record

    An uploaded clip can be read once into a typed record of people, PPE, objects and every ontology event. Later questions about typed events are answered from it with no model call.

  • Evidence gate

    Deterministic checks return one of five verdicts; a single blocking failure holds an answer whatever its score.

  • Question log

    Every answer shown is logged in the visitor's browser in a SHA-256 hash chain. It detects an edited entry; it cannot stop someone who recomputes every hash, which is why production anchors the chain server-side.

Production design, not yet built · 4

  • Edge appliance and camera ingestion

    Production keeps raw video on site. This web product sends masked stills to a third-party model, and says so beside every answer.

  • Role-based access and SSO

    Scoping questions and playback by role, with aggregate-only views for works-council members.

  • Server-anchored audit chain

    The question log's head written to write-once storage, so recomputing the chain is detectable.

  • Multi-site rollup and EHS integration

    Cross-plant questions, and held answers routed into the EHS system's corrective-action workflow.

Sample clips are stock footage, labelled as such, never customer or plant data. Their human labels are published on the Evidence tab.