Raspberry Pi 5 + Hailo-8L · 27 FPS · everything onboard
An attention-free student of Depth Anything V2 that runs on a 13-TOPS integer NPU with int8 equal to float, and the onboard obstacle avoidance, global planning and SLAM it makes possible on a small rover.
The Hailo-8L runs attention in int8, and diffuse attention at a few hundred tokens falls below one quantisation step: our build of Depth Anything V2 and the vendor's both erase obstacles. We distil the 335 M-parameter teacher into a 24.5 M-parameter network with no attention at all, range obstacles from where they meet the floor, and run mapping, hybrid-A* planning and SLAM on the Pi itself.
Demos
A low camera crossing a field toward rocks and posts. Every frame went through the student v5 HEF on the Raspberry Pi 5's Hailo-8L (34.6 ms per frame on the NPU) and was read back; the camera half is the 4K original. Frames from this video (every 10th) were in the student's training set, so it is a demonstration, not a held-out result. Download the 4K version.
Isaac Sim streams the car's camera to the real Pi; the Pi computes depth on the Hailo, maps, plans and replies with steering and throttle, applied only when they arrive. The six scenes one recorded pass completed with no contact, each uncut; that pass scored 6 clean and 7 reached of 10, the five-seed mean is 8.2.
Real robot video with depth computed on the Hailo-8L (student v3 HEF, every frame read back from the chip), the metric point cloud scaled by the floor, and the ground-contact scan with the chosen arc.
Every scene of the same pass, labelled with its outcome on the car's real footprint: chairs, a cone slalom, a barrel gate, people, mixed clutter and a crate wall, with timeouts where they happened.
cameraHailo-8L depth
Held-out real frames; depth from the deployed student v5, read back from the Hailo-8L.
Method
At the chip's 449 tokens, a diffuse softmax weight is about 1/449 ≈ 0.0022, below one uniform int8 step of 1/255 ≈ 0.0039. Measured in float, 91 % of attention weights fall below that step and carry 17–33 % of each query's attention mass. Re-calibration and quantisation fine-tuning do not fix it; removing attention does.
A ResNet-34 encoder and a U-Net decoder of convolution, batch norm, ReLU and bilinear upsampling: every operation quantises cleanly. 24.5 M parameters (DAv2-S: 24.8 M), distilled from DAv2-Large with a scale- and shift-invariant loss, then fine-tuned on rendered ground truth. On the chip: 35.1 ms, int8 = float at r 0.99993.
Relative depth has no scale. But where an obstacle meets the floor, the image row alone gives its distance from the camera height. The network only has to say where each column's floor ends; the columns become a 69°, 4 m laser scan, and the floor profile becomes a ruler for a metric point cloud.
Z = fy · h / (vc − cy)
Results
400 held-out simulated frames with the renderer's metric depth and 100 held-out real robot frames. Every Hailo model runs on the chip and is read back; per-frame least-squares scale and shift in disparity. Obstacle pixels are those that land on a prop's collider, which removes the renderer's floor speckle.
Hover or focus a point. Dashed lines: float GPU models that cannot run on the Pi at this rate.
| Model | Runs on | AbsRel all | AbsRel obst. | δ1 | r (real) | Speed |
|---|
Ten frozen scenes in a Gaussian-splat reconstruction of a real yard, eleven obstacle types with honest colliders, at least five seeds per arm, contacts scored on the car's real 0.557 × 0.294 m footprint. Clean goals of 10:
Execution costs about 0.6, perception 1.3, self-localisation 0.2 clean goals (0.9 goals reached). Ten seeds for the deployed arms, five for the others; the own-pose arms use a simulated wheel encoder and gyro.
Honestly
Resources
Citation
@misc{itaim2026seeing,
title = {Seeing to Drive on a Single Board: Attention-Free Distilled Depth on a
Raspberry Pi 5 + Hailo-8L for Onboard Obstacle Avoidance, Planning and SLAM},
author = {M., Itai},
year = {2026},
url = {https://itaim18.github.io/seeing-to-drive/}
}
Built on Depth Anything V2 and Depth Anything 3, the Hailo Model Zoo, NVIDIA Isaac Sim / Isaac Lab and its neural-reconstruction rendering. Weights and data CC-BY-NC-4.0 (the teacher's licence).