Train bipedal robot policies in simulation before deploying them to hardware
Imagine a small robot taking its first steps on the ground after undergoing thousands of trials and errors in a virtual space. This repository is a tool that automates that training process. It precisely replicates real-world factors such as joint friction and battery voltage drop, helping ensure that motions learned in simulation work exactly the same way in reality.
Precise modeling to bridge the gap between reality and simulation
The core of this repository is not a simple physics engine, but a simulation of the Dynamixel XL330 servo motors used in actual robots down to the electrical level. It includes voltage control laws, back-EMF, and friction that varies with load in its calculations. It also applies 1-degree backlash between gears to each joint, reflecting the fine errors that occur when the robot actually moves into the learning data. Thanks to this precise actuator modeling, the mismatch problem that occurs when a policy trained in simulation is applied to a real robot can be significantly reduced.
Training for various motions from walking to kicking
The repository includes more than 13 training tasks, such as walking on flat ground, walking on rough terrain, getting up after falling, standing up from a sitting position, kicking a ball, and rolling. Each task is provided in two versions: Flat and Rough, along with a variant that includes the backlash effect. All policies share the same 61-dimensional observation structure, allowing the robot to immediately switch from a walking policy to a recovery policy if it suddenly falls while walking. This policy hot-swap feature enables the robot to respond flexibly to various situations.
A single pipeline from training to deployment
In particular, using the uv run publish command allows trained policies to be uploaded to the Hugging Face Hub in a standardized format, enabling other users to install the policy on their own robot with a single line of robotctl policy add. During this process, the observation normalizer is automatically included in the ONNX graph, so it can be used immediately upon deployment without additional conversion steps. In environments without a GPU, training can also be run on Hugging Face Jobs using the --hf-jobs option.
Things to check before use
A CUDA GPU is required to use this repository. On ARM-based devices (such as DGX Spark and Jetson), downloading CUDA libraries during initial installation can take a long time, so it is recommended to set the UV_HTTP_TIMEOUT=600 environment variable. Training in 4096 environments takes approximately 1 to 2 hours to obtain a viable walk. Additionally, while this repository is released under the Apache 2.0 license, the 3D model files follow the Creative Commons BY-SA-NC license, so caution is needed for commercial use. Currently, only constant command-based policies can be deployed, while staged motions or posture flag-based motions are managed separately in the official policy repository.
