One of the frontiers of robotics research today is whole-body control of a humanoid robot like Asimov. Whole-body control unlocks a new level of capability for humanoid robots by allowing it to use all its limbs to solve problems, rather than having the upper body and lower body work as separate sub-systems. To measure the success of a whole-body control policy, it is common to look at how well a robot can replicate some pre-recorded whole-body motion in the physical world. The challenge of this is that the robot must be able to understand how to replicate the motion while balancing itself under gravity.
NVIDIA's SONIC is one of the best open source whole-body motion-tracking policies out there, and has become an industry standard starting point for research. An encoder compresses a short window of reference motion into a quantized motion token, and a decoder turns that token plus the robot's own state into joint targets. The public GEAR-SONIC checkpoints are 22.5M to 50M parameters.
We wanted to know if it was possible to deploy the SONIC model on using Asimov's existing compute capabilities.
It can. SONIC for Asimov, UFB's open-source port that keeps one encoder and the decoder, runs fastest: 2.6 ms per tick, or 4.2 ms on a single core. Full GEAR-SONIC also fits, including the 49.7M-parameter v1.1 at 9.4 ms.
Takeaways
- SONIC for Asimov (13.8M parameters) runs a full inference tick in 2.6 ms on the two ARM A76 cores of Asimov's RK3588, faster than the 50 Hz tick typically used for such control policies.
- Full GEAR-SONIC runs on the same CPU too: 4.5 ms for the original release and 9.4 ms for v1.1
- SONIC gets its own core, so the 200 Hz motor loop is untouched
- Pinning inference to its cores cut the v1.1 decoder's p99 from over 11 ms to 6.7 ms
What we ran
We ran UFB's SONIC for Asimov checkpoint and two full GEAR-SONIC releases for the Unitree G1, unchanged. Parameter counts are from the ONNX files.
| Model | Encoder | Decoder | Total | Output |
|---|---|---|---|---|
| SONIC for Asimov | 4.0M | 9.8M | 13.8M | 23 joints |
| GEAR-SONIC (original) | 12.3M | 10.2M | 22.5M | 29 joints |
| GEAR-SONIC v1.1 | 12.3M | 37.4M | 49.7M | 29 joints |
All three are MLPs with SiLU activations and an FSQ motion token between encoder and decoder.
- SONIC for Asimov is UFB's open-source port of SONIC to Asimov. Their sonic-asimov repo explains how it was trained. It keeps just the robot-motion encoder and drops the human-motion and VR encoders. That makes the encoder a third the size of GEAR-SONIC's, and most of the model is the decoder. The decoder has the same layer widths as the original GEAR-SONIC decoder, resized for Asimov's 23 joints (legs, waist and arms). Encoder and decoder are fused into one ONNX file.
- GEAR-SONIC (original) is the full release: the top-level
model_encoder.onnxandmodel_decoder.onnx. The encoder file holds three encoders for different reference inputs: robot motion, human motion and VR teleoperation. - GEAR-SONIC v1.1 uses the same encoder design with a much larger decoder: eight hidden layers starting at 4,096 wide, where the original has six starting at 2,048.
The Asimov model takes a reference motion window and the robot's recent proprioception, and outputs 23 joint position targets at 50 Hz. The motor drivers run PD around those targets. We did not run the low-latency variant or the kinematic planner.
The benchmark
We timed each model on the Radxa's CPU with ONNX Runtime. Each number is one full inference tick, encoder plus decoder.
| Model | Params | ORT defaults | 2 × A76 | 1 × A76 |
|---|---|---|---|---|
| SONIC for Asimov | 13.8M | 2.8 ms | 2.6 ms | 4.2 ms |
| GEAR-SONIC (original) | 22.5M | 4.8 ms | 4.5 ms | 6.9 ms |
| GEAR-SONIC v1.1 | 49.7M | 9.4 ms | 9.4 ms | 14.6 ms |
A 50 Hz policy has 20 ms per tick, and Asimov discards any inference that takes longer than 15 ms.
- SONIC for Asimov runs best. At 2.6 ms it uses about an eighth of the tick, and it is the one we are testing on the robot in the videos above.
- Full SONIC runs too. The original release takes 4.5 ms, and even v1.1 finishes in 9.4 ms on two cores. Most of that is the decoder, which takes 6.6 ms of v1.1's 9.4 ms tick.
So the board has room for full SONIC, not just a cut-down version. We would only avoid v1.1 on a single core, where 14.6 ms is too close to the 15 ms limit.
Fitting it next to the motor loop
The motor loop runs at 200 Hz, so it has 5 ms per cycle, and polling the CAN buses already takes about 4 ms of that. Asimov's smaller locomotion policy runs inline in that loop. SONIC can't: adding a 4.2 ms inference would make the loop late on every policy tick.
So SONIC runs in its own thread on its own core. It runs at 50 Hz and publishes its latest action. The motor loop reads that action each cycle and never waits for the network. A late result is discarded, and three in a row put the robot into damping.
The RK3588S has four A76 big cores and four A55 little cores, and Asimov's real-time Linux setup already assigns them:
- CPU7 (A76, isolated): 200 Hz motor loop and CAN interrupts
- CPU4 (A76, isolated): 500 Hz IMU fusion
- CPU6 (A76): SONIC inference
- CPU5 (A76): power monitoring
- CPU0–3 (A55): networking and Linux services
No kernel changes are needed. SONIC is one more
SCHED_FIFO thread with a fixed core and a priority below the motor and IMU threads.Pinning matters for the slow ticks
In a separate trace, the v1.1 decoder took 6.6 ms when pinned to its cores, with a p99 of 6.7 ms. Unpinned, the p99 went above 11 ms. The typical tick barely changed, but the slow ones got much slower. On a robot, the slow ticks set the budget.
What's next
We have started preliminary testing of SONIC for Asimov on the real robot, on a harness. The benchmark numbers cover inference only. Next we time the full path on the robot, from sensor reading to motor command, and take it off the harness.
SONIC's size turned out not to be the obstacle. The slimmed Asimov model is the fastest, but full GEAR-SONIC fits too. A whole-body policy fits on a small CPU-only computer inside the robot, as long as it gets its own core and the motor loop is left alone. Thanks to NVIDIA's GEAR team for releasing SONIC openly, and to UFB for open-sourcing the Asimov port.
The SONIC for Asimov model, reference motions and MuJoCo player are in UFB's sonic-asimov repo.
