> Documentation index: https://nao-rs.lukaloehr.com/llms.txt, a list of every page in this documentation.

# Troubleshooting

> What to check when nao-rs doesn't start, the death switch won't stay armed, the guard takes over, or a tool can't open the chest board.

Start with these two commands on the robot; most problems show up in one of
them.

```bash
naoctl status
journalctl -u hulk -u nao-rs-guard -n 50 --no-pager
```

## The motors don't move

That is expected today. nao-rs rejects the robot's version 11 motor boards,
as HAL 2.8.5 does, until the motor session on the robot (**planned**). See
[Status](https://nao-rs.lukaloehr.com/get-started.md#status).

## The switch is tripped and I didn't trip it

`naoctl status` names the reason. The first reason since the last re-arm is
kept.

| Reason | Cause | What to do |
| --- | --- | --- |
| `Cli` right after boot | Every boot starts with a new, tripped page | Re-arm when the robot is ready: `naoctl rearm` |
| `ChestButton` | The chest button was pressed | Check the robot, re-arm |
| `HeadTouch` | All three head pads were touched at once | Check the robot, re-arm |
| `Overtemperature` | A joint at 75 °C, or the CPU above 95 °C (the system halts then) | Let the robot cool down; check the fan |
| `DriverFault` | The cycle fell more than 3 periods behind, an exchange with the chest board failed, or the runtime stopped | Read `journalctl -u hulk`; look for `cycle overran` or `cycle failed` |
| `Api` | A stop from the REST API, its WebSocket, the MessagePack API or the brain's `stop` tool | Find out which client; re-arm |
| `Udp` | Someone sent the UDP stop alone (for example `cargo nao-rs stop`) | Find out who; re-arm |
| `GuardHeartbeatLost` | The runtime stopped beating for more than 60 ms | The guard killed `hulk`; see below |
| `PageLost` | The switch page in `/run/nao-rs` was truncated, overwritten, deleted or replaced | Find what touched `/run/nao-rs`. After a deletion or replacement, restart nao-rs (`sudo systemctl restart nao-rs-guard`, then `systemctl restart hulk`) so every stop path uses one page again; then re-arm |
| `GuardNoConfirmation` | After a trip, the runtime didn't confirm an all-off frame within 40 ms | The guard killed `hulk`; see below |

## The guard took over

`journalctl -u nao-rs-guard` shows `TakeOver`, `KilledDriver` and `Holding`.
The guard holds the chest board and sends all-off. systemd restarts `hulk`
after one second (`Restart=on-failure`); the guard hands the board to the new
`hulk`, and the switch stays tripped.

1. Read why `hulk` hung or died: `journalctl -u hulk -n 200 --no-pager`.
2. Check that the new `hulk` beats: `naoctl status`.
3. When the robot is safe, re-arm: `naoctl rearm`.

If `hulk` keeps failing, systemd gives up after repeated failures; fix the
cause and run `systemctl start hulk`.

## `hulk` doesn't start

- **`failed to start the robot runtime: chest board: …`**: another program
  holds the chest board. HAL may still run (`systemctl status hal`), or the
  guard holds it after a takeover (see above), or a `naoctl` hardware command
  runs.
- **`death switch: Permission denied`** or **`No such file or directory`**:
  `/run/nao-rs` is missing or has the wrong owner. It should be `root:hal`,
  mode `2770`. Run
  `sudo systemd-tmpfiles --create /etc/tmpfiles.d/nao-rs.conf`.
- **`head bus unavailable`** or **`head fan unavailable`** in the log: the
  runtime keeps running without the head or the fan. Check that LoLA is
  stopped (`systemctl status lola`) and that `/dev/i2c-head` and `/dev/cgos`
  exist.

## The installer rolled back

Exit code 1 means the health check failed and the previous installation runs
again. The installer printed the last 30 journal lines of both services; the
failed installation is kept in `/home/nao/hulk.failed`. Exit code 2 means the
rollback failed too: run `cargo nao-rs robot restore <address>`, or reflash.

## `naoctl` can't open the chest board

The hardware commands (`prefs`, `run`, `guard`, `head`, `replay-sensors`) need
the device to themselves. Stop `hulk` (`systemctl stop hulk`) and, for
`guard`, the guard service. On the original stack, stop HAL, and LoLA for
`head`. `status`, `stop` and `rearm` need nothing stopped.

## LoLA can't reconnect after a restore

On HULKs-OS, LoLA can't connect to a HAL that is already running ("No HAL
found"). Restart the whole chain in order: HAL, then LoLA, then HULA, then
`hulk`. `nao-rs robot restore` does this.

## The robot doesn't react to the UDP stop

- Check the guard runs: `systemctl status nao-rs-guard`.
- The datagram must start with exactly `NAO-RS STOP` and go to UDP port 7701.
- A firewall between you and the robot may drop UDP. On the robot,
  `naoctl stop` always works.

## The robot doesn't respond at all

Hold the chest button for 8 seconds: the chest board switches the robot off,
whatever runs on the head.

## Related topics

- [FAQ](https://nao-rs.lukaloehr.com/help/faq.md)
- Previous: [Command-line tools](https://nao-rs.lukaloehr.com/reference/cli.md)
- Next: [FAQ](https://nao-rs.lukaloehr.com/help/faq.md)