Detection is not understanding

A modern detector can tell you that a person is in zone B and that they are not wearing a hard hat. That is useful. But it is not the same as saying, "The loader in zone B is missing the required PPE for the pre-shift check." The second sentence is operational language: it connects the visual fact to the rule, the role, the process, and the moment.

Without that grounding, vision AI remains a dashboard. With it, the camera becomes an agent that participates in the workflow.

What we mean by grounding

Grounding is the bridge between two worlds: the continuous, noisy signal from the camera and the discrete, rule-based language of operations. It has three layers:

Spatial grounding. The agent knows where it is looking — zone B, hopper 3, the north loading bay — and can map detections to those named regions.
Temporal grounding. The agent knows what should happen when. A person in a restricted zone during a production run is different from the same person there during a scheduled maintenance window.
Semantic grounding. The agent knows the vocabulary of the site: SOPs, alerts, tickets, roles, shifts, and the causal relationships between them.

When all three layers are present, the agent can answer operator questions, raise exceptions in the language of the system of record, and explain itself in plain English.

From pixels to predicates

The raw output of a model is a tensor: coordinates, class probabilities, embeddings. We turn that into predicates that the operations layer can reason about:

Person(x) in Zone("B") at Time("08:14:23")
not Wearing(x, "hard_hat")
ActiveShift("morning") and SOP("pre_shift_check")

These predicates are not hand-written for every scene. They are generated from the site configuration: a floor map, a shift roster, a set of SOPs, and a catalog of PPE and equipment. The same vision stack can be deployed to a mine, a warehouse, or a retail store without changing the model. Only the configuration changes.

Operational language as the interface

Most operators do not want to write SQL or query JSON. They want to ask, "Why did the line stop at 3:12?" or command, "Alert me when the pallet wrapper runs without a strap." We are building a thin natural-language layer on top of the grounded predicate graph. The model does not need to know everything about the world; it just needs to know how to translate the operator's intent into a query over the grounded state.

Why this matters for trust

A vision system that only emits bounding boxes is a black box. A grounded agent can explain itself: "I flagged this because zone B is active during the pre-shift check, the person is not wearing a hard hat, and SOP-17 requires head protection in that zone while the conveyor is running." That is the kind of explanation a safety officer can act on. It is also the kind of audit trail that makes compliance straightforward.

The stack

Our grounding layer is built from a few primitives:

Scene graph construction. Every frame becomes a structured graph of objects, zones, and relationships.
Event extraction. The graph is diffed over time to produce events: entered, left, started, stopped, picked, placed, dropped.
Rule engine. Site-specific rules are evaluated against the event stream.
Language model. A small, local language model translates between the rule graph and natural language for operator queries and alerts.

The heavy vision work stays at the edge. The grounding and language work can run at the edge or on a small cloud node, depending on the latency and privacy requirements.

What comes next

Grounding is the step that lets vision agents move from passive observation to active participation. We are extending our grounding layer to handle maintenance workflows, quality-control checks, and even multi-agent coordination: one camera noticing a problem, another confirming the fix, and the system of record updating itself in between.

The goal is not to replace operators. It is to give them a colleague that never blinks, never forgets the SOP, and speaks the same language they do.