12ms at the edge: designing for constrained hardware
Why the most useful vision agents run on the cameras, not in the cloud — and how we design models that stay under 12ms on a factory edge node.
The camera is the computer
Most early computer-vision demos were built on the comfortable assumption of a GPU rack a few milliseconds away. Upload a frame, run a big model, get back a bounding box. For the demo it works. For a production line running three shifts at sixty frames per second, it does not. Bandwidth, cost, and latency all become structural problems the moment the camera count crosses a hundred.
That is why we design our agents to live inside the camera or the edge node itself. The cloud is for training, orchestration, and long-term analytics. The inference — the act of seeing and deciding — happens as close to the scene as possible.
What 12ms actually buys you
In a fast-moving line, a missed frame is a missed event. Twelve milliseconds leaves headroom for:
- Pre-processing the frame (debayer, resize, normalize).
- Running the model.
- Post-processing the output into structured events.
- Emitting a decision to the actuator or PLC.
It also lets us run multiple models on the same edge node without contention. Safety, quality, and tracking can share one box instead of three separate black boxes wired together.
The constraints we optimize around
Edge hardware is not a small cloud. It is a different species.
- Memory is measured in megabytes, not gigabytes. A large foundation model will not fit without heroic compression, and even then it will starve the rest of the system.
- Power is a first-class budget. Cameras in remote sites or on mobile equipment run on batteries or solar. Every multiply-add has a cost.
- Thermal matters. A fan in a dusty factory is a failure point. We design for passive heat sinks.
- Connectivity is intermittent. The edge node must keep operating when the network does not. The agent must cache, queue, and recover cleanly.
How we get there
We use a few techniques in combination:
1. Task-specific architectures
A model that can caption any image in the world is overkill for detecting whether a worker is wearing a hard hat. We start with smaller backbones, prune aggressively, and train on a narrow distribution. The result is a model that is faster, cheaper, and more accurate on the one thing it needs to do.
2. Quantization and graph surgery
FP32 weights are a luxury. We quantize to INT8 where possible, fuse operators, and eliminate dynamic shapes from the inference graph. We also hand-tune the runtime so that the critical path is one contiguous block of execution.
3. Sensor-level fusion
The camera sees the scene. A microphone or a vibration sensor can confirm it. A radar can do it in the dark. By combining low-bandwidth signals at the edge, we can avoid running a heavy model on every single frame.
4. On-device learning loops
The best model is the one that has seen your specific factory. We ship models that can do a small amount of fine-tuning on the edge node itself, using only labelled examples from your site, without sending raw video anywhere.
The hidden cost of latency
A 200ms cloud round trip does not just delay the alert. It changes the architecture of the system. Operations teams start adding buffers, second sensors, and manual checks. The AI becomes a screen people watch instead of a system that acts. We think the real win is not the model accuracy; it is the loop speed. At 12ms, the machine can close the loop with the physical world.
Looking forward
The next generation of edge silicon is adding dedicated neural-network accelerators, native support for transformer blocks, and low-power event-based sensors. We are building the software stack that will let these chips run agents, not just detectors. The camera will keep getting smarter, but the principle will stay the same: the right answer, in the right place, before the moment passes.