Skip to content

Command Messages

Command messages tell your robot what to do: set a mode, run a mission, apply a setting, push a zone, or stream teleop input. They flow from the Rover Nexus cloud → the on-robot agent → your robot software. Your robot software subscribes to the agent, not to the cloud directly.

  • Direction: Downlink (cloud → robot)
  • Topic: subscribe on the local Zenoh session to command (or command/** if per-type subtopics are enabled); see Receiving
  • Encoding: Cap'n Proto over Zenoh
  • Envelope: every command is a single AgentToApplication union

Local keys have no robot id. Your robot software subscribes to an un-scoped local key. The agent has already stripped the cloud routing (and the NexusCommand wrapper) and delivers the bare command on the local session.

This page documents every AgentToApplication variant. Shared building blocks (geometry, Value, spatial features, enums) live in Common Types.

Coordinate frames. Geometry-bearing commands (for example assignMission, updateFeature, spatialDirective, object) arrive in WGS84 by default. Set convert_commands_to_local = true in the robot configuration to receive that geometry in your robot's local ENU frame instead. velocityCmd and teleopJoy are body-relative and never converted. See Coordinate Frames.

How commands reach your robot

A command originates at the fleet manager. On the server → agent hop it is wrapped in a NexusCommand that carries a per-command UUID. The agent acknowledges the UUID (see RobotAck), persists assets it needs (missions, features, objects, settings), and forwards the inner AgentToApplication unchanged to your robot software. The NexusCommand wrapper is never forwarded; your software only ever sees a bare AgentToApplication.

Scheduled missions are held by the agent and fired when due.

The AgentToApplication envelope

AgentToApplication is a tagged union, so each command is exactly one variant. Time fields are Int64 Unix milliseconds.

struct AgentToApplication {
  union {
    setRobotMode @0 :RobotMode;  # set operating mode: auto, teleop, etc.
    pause @1 :Void;              # pause robot activity, motion, and the current mission
    resume @2 :Void;             # opposite of pause; resume the current activity or mission
    assignMission @3 :MissionCommand;  # a complete mission: path, target, actions, settings
    controlMissionRun @4 :ControlMissionRun;  # pause/resume/cancel a specific run
    invokeAction @5 :ActionCall;  # call an action the robot provides
    updateSettings @6 :List(SettingUpdate);  # set robot settings
    updateFeature @7 :FeatureOp;  # inform the robot of a feature in the world
    spatialDirective @8 :SpatialDirectiveOp;  # inform the robot of a field rule
    object @9 :ObjectOp;          # inform the robot of an object in the world
    velocityCmd @10 :VelTwist;    # set the raw trajectory of the robot (teleop only)
    teleopJoy @11 :TeleopJoy;     # joystick command for teleop
    agentTextRequest @12 :AgentTextRequest;  # text request for an on-robot LLM agent
    sayText @13 :SayTextRequest;  # request the robot say something
    messageConfirmation @14 :MessageConfirmation;  # operator confirmation of a robot message
  }
}
Variant Payload Purpose
setRobotMode RobotMode Set the operating mode (auto / teleop / …).
velocityCmd VelTwist Direct forward/angular velocity.
pause (void) Pause activity and the current mission.
resume (void) Resume the paused activity/mission.
invokeAction ActionCall Invoke an advertised action.
assignMission MissionCommand Run a complete mission.
controlMissionRun ControlMissionRun Pause/resume/cancel a specific run.
updateSettings List(SettingUpdate) Apply one or more settings.
updateFeature FeatureOp Upsert/delete a bare spatial feature.
spatialDirective SpatialDirectiveOp Upsert/delete a zone with rules.
object ObjectOp Upsert/delete a tracked object.
teleopJoy TeleopJoy Teleop joystick state.
agentTextRequest AgentTextRequest Text request for an on-robot LLM agent.
sayText SayTextRequest Ask the robot to speak.
messageConfirmation MessageConfirmation Operator confirm/deny of a robot message.

No remote e-stop. There is intentionally no remote-EStop command. Emergency stop is a physical safety function and must not be triggered over the wire. Use pause for a remote pause/halt. Robots still report their physical e-stop state via statusTelemetry.estop.

No navigateTo command. The "drive to this pose" command was removed. Send a destination as a Feature with layerRole = target (via updateFeature), or as a mission whose feature is the route or target to reach.

Motion and mode

setRobotMode: change operating mode

Set the robot to an operating mode. The payload is a RobotMode enum.

setRobotMode @0 :RobotMode;  # manual | auto | teleop | disabled | maintenance

velocityCmd: direct velocity

Set the raw trajectory of the robot. Typically used for low-level/closed-loop control where the cloud (or an API client) drives velocity directly. The payload is a VelTwist.

velocityCmd @10 :VelTwist;  # forwardMps, angularRadps

pause and resume

Unit (void) variants. pause halts robot activity and pauses the current mission; resume is its opposite, resuming the current activity or mission. These are a remote pause/halt, not an emergency stop. A robot cannot opt out of receiving them (see allowedCommands).

pause @1 :Void;
resume @2 :Void;

Actions and settings

invokeAction: call an action

Invoke an action the robot advertised in its capabilities manifest. For a trigger action, leave setting unset; for a setBool action, setting is the desired on/off state.

struct ActionCall {
  actionName @0 :Text;
  setting @1 :Bool;
  hasSetting @2 :Bool;
}

The robot may report the resulting state change via the action's associated stateKey in sensorTelemetry.

updateSettings: change settings

Set one or more robot settings. Each entry must be a setting the robot declared in its capabilities manifest or its config. This is always a patch and is applied in order (not idempotent).

updateSettings @6 :List(SettingUpdate);

Each SettingUpdate is a key plus a typed Value. Confirm the applied values by watching currentSettings.

Missions

assignMission: deploy a mission

A complete mission order from the fleet manager: one asset, one-time use. It may carry mission parameters (settings applied at the start), actions to fire or enable, and at most one spatial Feature (a path, waypoints, or coverage area).

struct MissionCommand {
  missionRunId @0 :Text;  # mission assignment UUID, given by the server
  name @1 :Text;          # display name
  scheduledStartMs @2 :Int64;  # epoch ms
  missionParameters @3 :List(SettingUpdate);  # settings applied at the start of this step
  actions @4 :List(MissionAction);  # what actions this mission should use or call
  feature @5 :Feature;    # optional; path to follow, waypoints, or coverage area
}

# An action to fire or enable during a mission step, with timing.
struct MissionAction {
  name @0 :Text;
  timing @1 :ActionTiming;
  value @2 :Bool;
}
Field Type Notes
missionRunId Text The run id, assigned by the server. Echo this back in missionRunStatus.
name Text Display name.
scheduledStartMs Int64 Epoch ms. A future time is scheduled by the agent; a past time runs immediately.
missionParameters List(SettingUpdate) Settings applied at mission start.
actions List(MissionAction) What to fire/enable, with ActionTiming.
feature Feature Optional route/area for the mission.

One id per run. A mission run is identified by a single missionRunId. The old missionId / runId pair is gone, as are userId, detail, and expectedEndTimeMs (the robot reports its own estimate on missionRunStatus).

Scheduling behavior:

  • A mission with a future scheduledStartMs is scheduled by the agent and fired when due, carrying the same missionRunId.
  • A mission with a past start time is forwarded immediately.
  • Missions fired too far past their start time may be skipped.

Report progress and the final outcome with missionRunStatus.

controlMissionRun: control a run

Apply a control action to a specific mission run: pause, resume, or cancel it. This controls the mission lifecycle, not overall motion (use pause / resume for motion). cancel also removes the mission from the robot.

struct ControlMissionRun {
  missionRunId @0 :Text;
  action @1 :MissionRunAction;
}

enum MissionRunAction {
  pause @0;
  resume @1;
  cancel @2;
}

abort is gone. Use cancel to stop and remove a run. A run the robot ends on its own can still report the aborted status.

Spatial assets

These three commands push world knowledge to the robot. All use upsert/delete operation wrappers with idempotent upserts (re-sending an id overwrites; delete removes by id). The agent persists these so the robot can recover them offline and via queryables.

updateFeature: geometry only

FeatureOp carries a bare Feature (geometry plus identity and classification): no timing or rules. Use spatialDirective when the payload also carries timing/parameters/actions for a zone.

struct FeatureOp {
  union {
    upsert @0 :Feature;
    delete @1 :Text;  # id to delete
  }
}

A feature with layerRole = target is how the cloud tells a robot where to go outside of a mission; there is no separate "navigate to" command.

spatialDirective: zone with rules

SpatialDirectiveOp carries a SpatialDirective: a persistent zone, namely a Feature plus timing, zone parameters, and the action names to apply while in or at the zone. The robot reports whether it is applying a field rule via spatialDirectiveStatus.

struct SpatialDirectiveOp {
  union {
    upsert @0 :SpatialDirective;
    delete @1 :Text;  # id to delete
  }
}

object: push a tracked object

ObjectOp carries a tracked Object (a person, vehicle, obstacle, …). The cloud can push objects the robot should know about; the robot can also report objects it detects (see object uplink).

struct ObjectOp {
  union {
    upsert @0 :Object;
    delete @1 :Text;  # id to delete
  }
}

Teleoperation

teleopJoy: teleoperation input

Teleoperation joystick state sent from the cloud to the robot. Your robot software subscribes to the command topic and applies it to motion control. sessionId is unique per teleop takeover/session.

# Sources: "webrtc-ui", "local-gamepad", "api-client:<id>".
struct TeleopJoy {
  sourceId @0 :Text;
  sessionId @1 :Text;
  axes @2 :List(Float32);
  buttons @3 :List(Bool);
  timestampMs @4 :Int64;
}

Tip. Treat teleop as a safety-critical, latency-sensitive stream. Apply a watchdog: stop the robot if teleop samples stop arriving. See Teleoperation for the full session flow and video.

Operator interaction

These commands let an operator talk to the robot and respond to its prompts. A robot advertises which it accepts via allowedCommands.

agentTextRequest: text to an on-robot agent

A free-text request from the operator, intended for robots that run an on-robot LLM agent. It communicates intent only; the robot must enforce operational safety and is recommended to sanitize inputs. If responseRequired is set, the robot must reply with a message uplink.

struct AgentTextRequest {
  text @0 :Text;
  intent @1 :AgentTextIntent;
  responseRequired @2 :Bool;
}

enum AgentTextIntent {
  observeAndReport @0;        # retrieve data, then respond
  answerQuestion @1;          # do not take action, simply answer
  operatorInstruction @2;     # operator asks the robot to take action
  troubleshootingRequest @3;  # operator requests troubleshooting / diagnosis
  generalMessage @4;          # anything else from operator to robot
}

sayText: speak text

Ask a speech-capable robot to say something. priority is intent and must be enforced by the robot.

struct SayTextRequest {
  text @0 :Text;
  priority @1 :SpeechPriority;
}

enum SpeechPriority {
  low @0;        # say this at the robot's next convenience
  next @1;       # say this ASAP
  interrupt @2;  # interrupt current speech
}

messageConfirmation

The operator's confirm/deny reply to a robot message that was sent with needsConfirmation. confirmationId echoes the id from that message.

struct MessageConfirmation {
  confirmed @0 :Bool;
  confirmationId @1 :Text;
}

NexusCommand and acknowledgements

These types exist only on the server → agent hop and the agent's status reporting back to the server. Your robot software never sees them. They are documented here so integrators understand the command lifecycle and the acks the agent emits on your behalf.

# Server -> agent wrapper carrying a per-command UUID (16 raw bytes on the wire).
struct NexusCommand {
  id @0 :Data;
  command @1 :AgentToApplication;
}

# Lifecycle of a NexusCommand.
enum AckStatus {
  sent @0;
  agentReceived @1;
  agentRejected @2;
  robotReceived @3;
  robotRejected @4;
  robotStateConfirmed @5;
}

# Agent/robot -> server status report for a previously sent NexusCommand.
# commandId echoes NexusCommand.id.
struct RobotAck {
  commandId @0 :Data;
  status @1 :AckStatus;
  reason @2 :Text;  # optional
  receivedAtMs @3 :Int64;
}

NexusCommand and RobotAck are defined in a separate agent ↔ server schema (server_agent_messages.capnp), not in the robot-software schema (messages.capnp). On the cloud link every agent → server frame is one AgentToServer union and every server → agent frame is one ServerToAgent union; the RobotAck above and the connectivity heartbeat ride inside them. None of this is part of your robot-software contract.

Error handling

When a command arrives, robot software should:

  1. Validate the command before executing it.
  2. Report errors via a fault or message uplink.
  3. Update missionRunStatus when mission-related commands are processed.

Receiving

Subscribe on the local Zenoh session and deserialize each payload from Cap'n Proto into an AgentToApplication. By default the agent publishes every command to the single base topic command. Subscribe there and branch on the AgentToApplication union discriminant. The local key does not include the robot id.

Optionally, individual command types can be given their own subtopic in the robot configuration so they fan out onto distinct topics. A command whose subtopic is not configured stays on the base topic; when a subtopic is configured the command publishes to {base}/{subtopic} (e.g. base command + velocity → command/velocity), so subscribe to command/** to receive them all. Any of the command variants above (setRobotMode, velocityCmd, pause, resume, invokeAction, assignMission, controlMissionRun, updateSettings, updateFeature, spatialDirective, object, teleopJoy, agentTextRequest, sayText, messageConfirmation) can be split out this way.

  • Native: subscribe directly and act on the variant. See Custom Integrations.
  • ROS 2: the bridge delivers these commands onto your ROS 2 topics.