Verification and parity
How each part of nao-rs is proven to behave like the software it replaces - differential tests against the original libraries, replays of real recordings, and live checks on the robot.
"Ported 1:1" means: the same bytes on the wire, in the same order, with the same timing and the same reactions to errors, as Aldebaran's HAL and LoLA (NAOqi 2.8.5, as shipped in HULKs-OS 9.0.1). It doesn't mean copying their internal object model. Every part passes up to three gates before it counts.
The three gates
| Gate | How |
|---|---|
| Reference | A 32-bit test program loads Aldebaran's original libraries from the robot, with the robot's own loader, on a Linux PC and calls the original functions. A differential test feeds the same random inputs to the original and to the Rust port and compares every output byte |
| Recording | A recorder preloaded into HAL and LoLA on the robot logs every chest board USB transfer and every head I2C access, without changing anything. The port is then replayed against the recording call by call: it must send exactly the recorded requests when given the recorded answers |
| Robot | The final check on the NAO, with HAL, LoLA and HULA stopped |
Each differential test is checked for its own power with planted bugs: a mutated port must fail it.
Results
Updated 2026-11-06.
| Part | Reference | Recording | Robot |
|---|---|---|---|
USB link to the chest board (usbfs, chestlink) | Call-by-call replay of HAL's recording | Start-up and the 4,635-byte preference read in 275 ms | |
Chest board frames (chestboard) | 2,000,000 random cases, 0 mismatches | Every exchange | |
Motor board configuration table (motorboard) | 3,000,000 random cases, 0 mismatches | Version 11 answers decoded | |
Body preferences (chestprefs) | Replay identical to HAL; 76 of 76 values equal HAL's log | Values identical to HAL's | |
Body start-up, handshake, cycle frames, all-off (body) | Full HAL logs of two recordings byte-exact: 23,904 USB calls | HAL's handshake; version 11 boards rejected exactly as HAL does | |
Body sensors (sensors) | 279,186 values in 1,767 LoLA messages, bit-identical | IMU, battery, sonar tracking a person | |
Head bus (head) | 4,313 I2C transactions in two recordings, byte-identical | Eyes red, green, blue; 24.8 V, 10.7 W, 43 to 47 °C; 5.8 ms per transfer | |
Safety rules (safety) | Controller lost: 40,000 random cases (30 million steps); battery over-current: 50,000 cases (50 million steps); 0 mismatches | ||
Fan control (fan) | Control law identical on 300,000 configurations, 60.1 million steps | Board QA32, 52 °C, speed set and read back | |
Death switch (deathswitch, robot) | Tests, incl. a real process killed by the guard, 30 of 30 runs; 2,000,000 concurrent heartbeat reads | Frozen runtime taken over (62 ms for hulk), first all-off frame 9 ms after the kill; chest button hard power-off at 8 s | |
Runtime (robot, rt) | Tests with fake hardware | 1,043 cycles in 15 s, slowest 12.4 ms, stiffness 0 throughout | |
Chest board tools (chesttools) | Request sequences checked against the originals | chest-version replayed against HAL's start-up in both recordings |
Mutation checks
| Test | Planted bug | Cases that caught it |
|---|---|---|
| Configuration table, write | One wire byte written as 1 | 19,410 of 20,000 |
| Configuration table, read | One byte skipped | 19,324 of 20,000 |
| Current calibration | One table value changed | 71 of 20,000 |
| Frame decoder | Wrong acknowledge byte and an off-by-one in the length table | 14,820 of 20,000 |
Still open
Needs the motor session on the robot (planned):
- What an accepted version 11 board receives after the handshake, and that the boards obey it.
- Joint positions, currents, temperatures and hand touch with real data; the position glitch filter and the status bits of the motor boards; the joint zero positions applied.
- What the motor boards do when frames stop: the board-level behaviour behind the death switch.
- That the boards obey the guard's takeover frames.
Checkable without motors:
- The order of the head touch pads, the foot pressure sensors and the bumpers: one touch each.
- Whether the gyroscope's zero offset is ever nonzero.
Bugs found on the way
The ports keep the original's behaviour on the wire. Where the original is unsafe inside our own process, the port is stricter:
| In the original | In nao-rs |
|---|---|
| The frame decoder writes more than 255 message entries without a bound check, an overflow with hostile frames | Stops at 255 |
| The configuration table writer checks room for 2 bytes and then writes 4 | Same bytes inside the buffer, never past its end |
| HAL turns a NaN position into a count of 0 on the wire | The joint goes limp |
| alfand with a missing or broken settings file halts the system at the first reading above 0 °C | The file is refused |
| LoLA never resets the head power monitor and shifts its configuration fields wrongly | Kept on the wire, documented |
| LoLA reads head voltage, power, current and temperature but never sends them | Reported |
| HULKs-OS never starts the fan daemon, so nothing controls the head fan | The runtime runs the fan curve |
| The battery temperature is scaled twice (28.5 °C reads 2.8) | Kept in the runtime and the MessagePack API for compatibility; the REST API reports whole degrees Celsius |