Voice agent

Create your own ElevenLabs agent for the robot, pick the realtime model, put nao-rs's client tools on it, and what each tool does and refuses.

The conversation runs on your own ElevenLabs conversational agent. Its prompt, voice, language and model are set at ElevenLabs; nao-rs never changes them. The brain needs only the agent's ID, and for agents with authentication on, your API key.

Create the agent

Create a blank agent

At elevenlabs.io/app/agents, create a blank agent. Write its prompt, pick a voice and a language. The nao-rs repository has an example prompt in docs/examples/agent-prompt.md. Tell the prompt about the brain's context notes in your language: [Perception] and [Feeling] in English, [Wahrnehmung] and [Gefühl] in German (see Settings).

Pick the model and the audio format

Setting	Value	Why
Model	the realtime model `eleven_realtime_v1_max`	Answers in 0.6 to 0.9 s; a cascaded LLM such as Gemini took 2 to 7 s in our measurements
Output and input audio format (Voice, Advanced)	PCM 16000 Hz	The robot's own format. Other PCM rates and µ-law 8000 Hz are converted by the brain; MP3 and Opus don't work

Give the brain its ID

Copy the agent ID (agent_…) from the agent's page and set it in one of three ways: --agent-id, NAO_RS_AGENT_ID, or agent_id in ~/.config/nao-rs/brain.toml (see Settings).

If the agent has authentication on, the brain also needs your API key: from XI_API_KEY or ELEVENLABS_API_KEY, or from a command named in api_key_command. The key never goes into the config file and is never logged. A public agent needs no key, but anyone who knows its ID can talk to it on your account.

Put the tools on the agent

cargo nao-rs agent tools <agent-id>              # agent mode: all tools
cargo nao-rs agent tools <agent-id> --mode live  # live mode: halt, stop and memory only
cargo nao-rs agent tools <agent-id> --dry-run    # show what would change

This needs your ElevenLabs API key (ELEVENLABS_API_KEY, or the macOS Keychain item elevenlabs-api-key). It adds or updates only nao-rs's tools, by name, leaves every other tool on the agent alone, and never changes the prompt, voice or model: it sends the prompt back unchanged and checks it afterwards. Run it again after updating nao-rs.

Tools

The agent calls ElevenLabs client tools; the brain checks each call, checks the robot's state, sends the robot API commands and answers with what the robot does now, or why not. Every tool is asynchronous: the agent says what it does and keeps talking while the robot acts. Every call and its result appear in the brain's log. The definitions live in tools/brain/agent-tools.json in the nao-rs repository, the same file the brain checks calls against.

No motion in this phase

nao-rs doesn't drive the motors yet, and the robot must not move during bring-up without a person present. The motion tools are listed so you can set up an agent; on nao-rs today they change hulk's state without moving a joint.

Tool	Arguments	What the robot does	Modes
`walk`	`meters` (−1 to 1, negative: backward), optional `sideways` (positive: left)	Walks there and stops, keeping its heading. Longer walks are capped at 1 m and the agent is told so	agent
`turn`	`degrees` (−360 to 360, positive: left)	Turns on the spot; more than 180° in two parts	agent
`circle`	optional `direction` (`left`, `right`)	Turns once all the way around in four quarter turns, then stands	agent
`arms_up`	none	Raises both arms for 3 s, then stands	agent
`squat`	none	Squats with raised arms for 3 s, then stands up	agent
`sit_down`	none	Sits down	agent
`stand_up`	none	Stands up and stands still (`stand` is accepted too)	agent
`look`	`direction`: `ahead`, `left`, `right`, `up`, `down`, `around`	Turns the head; only while standing	agent
`turn_head`	`degrees` (−90 to 90, positive: left)	Turns the head sideways; only while standing	agent
`eyes`	`color` (`red`, `orange`, `yellow`, `green`, `cyan`, `blue`, `purple`, `pink`, `white`, `off`, `default`), optional `seconds` (0.1 to 60)	Colours both eyes, for a while or until changed; `default` gives them back to the robot	agent
`halt`	none	Stops whatever the body does and stands still	every mode
`stop`	none	Emergency stop: trips the death switch. All motors off at once; the robot goes limp and may fall. Someone re-arms it on the robot	every mode
`learn_name`	`name`	Remembers the name of the person in front of it	every mode
`remember`	`fact`	Remembers something about the person	every mode

Waving is not possible yet: the robot API has no arm motion besides arms up and squat.

Safety rules of the tools

Arguments are checked before anything moves. Wrong types, missing values and out-of-range numbers are refused with a message the agent can act on.
Motion is refused while the robot rests (motors off), falls or gets up, and head movements while it sits (they would stand it up).
A refused call stops a walking robot and cancels the running plan; a sitting robot stays sitting.
halt and stop cancel everything and go out at once. stop can't be undone from the Mac.
The robot's own safety comes first: fall protection, the watchdog that sits it down when the Mac goes away, and the death switch.