Research

Running SONIC on Asimov

NVIDIA's SONIC whole-body motion policy runs at 50 Hz on the CPU inside Asimov, next to its 200 Hz motor loop. The slimmed SONIC for Asimov model runs fastest, and full GEAR-SONIC fits too.

How we achieved zero-shot sim2real for Asimov

Policies trained in simulation now run on Asimov zero-shot, across multiple units with no per-robot tuning. How we closed the locomotion sim2real gap.

From walking to flipping: Porting our RL stack from locomotion to manipulation

We expected teaching Asimov to flip cartons would require starting over. Instead, lessons from locomotion shaped our manipulation pipeline, and much of the underlying RL machinery remained reusable. Here is what carried over, what changed, and why.

Kepler v0.1: A world encoder that learns physics across a robot's senses

Kepler-Encoder v0.1 is a self-supervised world encoder that fuses vision, proprioception, and force/torque into one shared representation, so a robot's senses teach each other and vision alone recovers force and body state at test time.

Noise is all you need to bridge the sim-to-real locomotion gap

We bridged the sim-to-real gap for Asimov's locomotion by running the real robot firmware and control stack inside a deliberately imperfect, noisy simulation.

Teaching a humanoid to walk: The RL policy behind Asimov's first steps

How we trained the reinforcement learning policy that gave Asimov its first steps.

How we built humanoid legs from the ground up in 100 days

We built a full set of humanoid legs from scratch, and got them to walk all in under 100 days. We're open sourcing everything.

Enabling GPUDirect P2P in OpenStack VMs

How to enable NVIDIA GPUDirect P2P in OpenStack virtual machines using a QEMU wrapper script. Achieve up to 137% higher GPU bandwidth and 97% lower latency.

AlphaSpace: A step closer towards having clumsy-less robots

AlphaSpace teaches language models to navigate and manipulate objects in 3D space through a specialized tokenization approach, bridging language and physical action.

VoxRep: Teaching a 2D model to see in 3D

VoxRep is a lightweight framework that lets a standard 2D vision-language model read 3D voxel scenes by slicing the grid into image tiles like a CT scan, recovering object location, color, and size without any specialized 3D architecture.

PoseLess: Vision-based robotic hand control without pose estimation

PoseLess is a groundbreaking framework that revolutionizes robot hand control by eliminating the need for explicit pose estimation. By directly mapping 2D images to joint angles using projected representations, PoseLess enables more intuitive and efficient control of robotic hands.

AlphaMaze: Teaching LLMs to think visually

AlphaMaze is a decoder-only LLM trained with GRPO to solve maze puzzles, improving the visual and spatial reasoning that ARC-AGI-style tasks demand.

Llama3 just got ears

llama3-s v0.2 is our latest open multimodal checkpoint, with stronger speech understanding across benchmarks.

Can Llama learn to listen?

We're teaching Llama 3 to understand speech: an early look at llama3-s, our open, early-fusion experiment trained on synthetic audio for under $600.

🍓 Ichigo: Llama learns to talk

With the latest checkpoint, we're teaching AI to talk and recognize when it can't listen.