Questions
![]()
QR code for Google Sheet
SimR
Simulation in Robotics
What simulator should I use?
That question is common in robotics research, robot learning, planning, control, and deployment.
The harder question is usually:
What simulator should I use for this robot, this task, and this research question?
Advise
- Stick with whatever makes you productive (or happy).
- MuJoCo (with MJWarp) is pretty stable.
- Isaac Sim has a lot of features.
- URDF is ubiquitous.
- This tutorial
- Newton Physics
- USD (and URDF)
- Viser Viewer
About Me
I am a professor in the Computer Science Department at Pomona College in Southern California, USA.
I learned a lot about simulation the hard way; you shouldn’t have to.
My Simulation Problem
I’ve spent arguably too much time trying different simulation methods and tools:
- Player stage, Webots, Gazebo, etc.
- Simulink/MATLAB, Julia, SimPy, FEA
- ODE, Bullet, DART, Box2D, Chrono, MuJoCo
- SAPIEN, ManiSkill, Newton Physics
- Gazebo, Webots
- Unreal, Unity, Godot
- custom simulators
- and many more
At ICRA, there are always several more that I will be tempted to explore.
Simulation in Robotics
- Have some idea of a physical device
- Create a simulation
- Run system identification and calibration experiments
- Optimize and experiment in simulation
- Transfer to the real world
- Back to previous steps…
Robotics Simulation
Simulation is the use of a model to study, predict, test, train, or evaluate the behavior of a robot or robotic system.
In robotics, simulation is used for:
- prototyping
- control design
- planning
Robotics Simulation (cont.)
- robot learning
- perception
- benchmarking
- regression testing
- sim-to-real transfer
- hardware/software integration
- safety and failure analysis
Simulation Is Not One Thing
A simulator may model:
- robot kinematics
- rigid-body dynamics
- contacts and friction
- actuators and transmissions
Simulation Is Not One Thing (cont.)
- sensors and perception
- lighting and rendering
- terrain, fluids, or deformable objects
- communication and latency
- uncertainty and noise
- humans, objects, or other robots
The right simulator depends on which of these matter for your question.
Research Questions (1/2)
Robotics simulation supports application-focused questions:
- How can we train a robot policy safely and efficiently?
- How can we evaluate a controller before deploying it?
- How can we test a planner across many environments?
- How can we compare methods under controlled conditions?
- How can we identify likely failure modes before hardware testing?
Research Questions (2/2)
Robotics simulation also supports method-focused questions:
- How accurate does the simulation need to be?
- Which modeling assumptions matter?
- What transfers from simulation to reality?
- How should we randomize or calibrate the simulator?
- How do we make simulated experiments reproducible?
- How do we avoid overfitting to a simulator?
Live Demos
Now let’s see some of these questions in action with live demos.
To the tutorial
A Common Robotics Simulation Loop
flowchart LR
Model[model robot and environment] --> Sim[simulate]
Sim --> Eval[evaluate behavior]
Eval --> Analyze[analyze failures]
Analyze --> Improve[improve model, controller, planner, or policy]
Improve --> Sim
Eval --> Deploy{deploy?}
Deploy --No--> Improve
Deploy --Yes--> Real[test on hardware]
Real --> Compare[compare sim and real]
Compare --> Improve
A Common Experiment Loop
config = load_config()
sim = make_simulator(config)
robot = load_robot_model(config)
task = load_task(config)
policy = load_policy(config)
for seed in seeds:
env = make_environment(sim, robot, task, seed)
for episode in range(num_episodes):
trajectory = rollout(env, policy)
result = evaluate(trajectory, task)
save_result(seed, episode, result)
save_trajectory(seed, episode, trajectory)
summary = analyze_results()
make_plots(summary)
if ready_for_hardware(summary):
run_real_robot_experiment(policy)
Optimization and Learning Loop
Some robotics projects use optimization, reinforcement learning, imitation learning, or evolutionary search.
flowchart LR
Init[initialize policy, controller, or design] --> Eval[evaluate in simulation]
Eval --> Stop{stop?}
Stop --Yes--> Retn[return best candidate]
Stop --No--> Update[update candidate]
Update --> Eval
Python Interface Sketch
Observation = list[float]
Action = list[float]
Trajectory = list[tuple[Observation, Action]]
Metrics = dict[str, float]
Policy = object
def make_simulator(config): ...
def load_robot_model(config): ...
def reset_environment(seed: int): ...
def policy(obs: Observation) -> Action: ...
def rollout(env, policy: Policy) -> Trajectory: ...
def evaluate(trajectory: Trajectory) -> Metrics: ...
Simulation
This section and the related demos are the focus of this tutorial.
Types of Simulation
- Analytical or closed-form dynamics
- Numerical dynamics
- Rigid-body dynamics
- Contact and friction simulation
- Sensor simulation
- Rendering and perception simulation
- Soft-body dynamics
- Finite-element analysis
- Digital twins
Common Robotics Simulators
Examples include:
- Gazebo
- MuJoCo
- Drake
- Webots
- CoppeliaSim
- NVIDIA Isaac Sim / Isaac Lab
- PyBullet / Bullet
- RaiSim
- Project Chrono
- Simscape Multibody
- Unity- or Unreal-based environments
The goal today is not to declare one winner.
The goal is to understand tradeoffs.
Simulation Advice (1/4)
Choose the correct level of abstraction.
- What do you care about?
- Are you studying perception, planning, control, learning, design, or deployment?
- Do you care about the electro-mechanical properties of the motors?
- Do you need accurate contact and friction?
- Do you need actuator saturation, latency, backlash, or compliance?
- Do you care about vision, depth, LiDAR, tactile sensing, or force sensing?
Simulation Advice (2/4)
Simplify when possible.
- Can you simplify the robot geometry?
- Can you approximate the wheels, gripper, or contact surfaces?
- Can you use kinematics instead of dynamics?
- Can you use simple rendering instead of photorealistic rendering?
- Can you approximate hydrodynamics, deformable objects, or terrain?
- Can you test the algorithm in a toy environment first?
The best simulator is often the simplest simulator that captures the phenomenon you care about.
Simulation Advice (3/4)
Consider your simulation characteristics.
- Simulation should be noisy enough to expose brittle solutions.
- Simulation should be deterministic enough to debug.
- You should evaluate a policy, controller, or planner with multiple initial conditions.
- Your simulation should usually be faster than real time.
- Your simulation should be parallelizable.
- Your simulation should be able to run headless.
Simulation Advice (4/4)
Numerical details matter.
- Avoid unnecessarily stiff springs.
- Avoid unrealistic controller gains.
- Use reasonable masses, lengths, forces, and time steps.
- Scale your environment to reasonable numerical values.
- Use fixed time steps when appropriate.
- Log simulator warnings and instability.
- Test extreme parameters and edge cases.
The Reality Gap
The main challenge is the reality gap.
Simulation and reality differ because of:
- simplified dynamics
- incorrect inertial parameters
- contact and friction errors
- unmodeled actuator behavior
- sensor noise and bias
- latency
- calibration errors
- lighting and rendering differences
- object and terrain variation
- wear, damage, and hardware variability
Sim-to-Real Advice
Think about sim-to-real from the beginning.
- Decide what needs to transfer.
- Measure physical parameters when possible.
- Calibrate the simulator with real data.
- Use domain randomization when appropriate.
- Randomize initial conditions, object properties, terrain, sensors, and actuation.
- Prefer robust policies over policies that exploit one narrow simulated setting.
- Validate intermediate quantities, not just final task success.
Decouple the Pipeline
Decouple simulation, visualization, learning, and evaluation.
- Make it easy to swap out simulators.
- Make it easy to swap out controllers, planners, policies, or optimizers.
- Make it easy to swap out visualization tools.
- Separate training from evaluation.
- Separate robot models from task definitions.
- Separate configuration from source code.
Avoid Simulator Overfitting
Be careful that your system is not too easy to game.
Policies and controllers may exploit:
- reward bugs
- termination conditions
- contact artifacts
- sensor artifacts
- unrealistic friction
- unstable numerical integration
- privileged simulator information
- task distributions that are too narrow
If it looks too good, inspect the rollout.
Robotics Experiment Advice (1/3)
Always prototype.
- Start with the simplest robot, task, and environment.
- Test controllers with simple benchmark systems.
- Test planners with simple maps.
- Test policies in small environments.
- Test simulations with known behaviors.
- Test extreme parameters.
- Add complexity one piece at a time.
Robotics Experiment Advice (2/3)
Save enough information to understand what happened.
- Save configurations.
- Save random seeds.
- Save simulator versions.
- Save robot models.
- Save trained policies or controller parameters.
- Save evaluation results.
- Save trajectories, logs, and videos when possible.
- Save failures, not just successes.
Robotics Experiment Advice (3/3)
Plan for reproducibility.
- Use checkpointing.
- Use structured output directories.
- Use configuration files.
- Track dependencies and software versions.
- Record hardware information when relevant.
- Use tools like Weights & Biases, MLflow, Sacred, or simple CSV/JSON logs.
- Make it possible for future you to rerun the experiment.
Evaluation Advice
Evaluate beyond the training conditions.
- Use held-out environments.
- Use held-out objects, terrains, lighting, or initial states.
- Use multiple random seeds.
- Report variability.
- Compare against baselines.
- Compare against simple hand-designed controllers or planners.
- Report wall-clock time, sample efficiency, and compute when relevant.
Parallelism (1/3)
Consider your computational resources.
Parallelize across:
- random seeds
- training runs
- evaluation episodes
- environments
- policies, controllers, or planners
- robot designs
- hyperparameter settings
Parallelism (2/3)
Useful tools include:
Parallelism (3/3)
Rules of thumb:
- Plan to leave a few cores free for the operating system.
- Parallelize at the random-seed level when you care about variability.
- Parallelize at the rollout or environment level when evaluations are expensive.
- Profile before optimizing.
- Avoid manual experiment management when possible.
- Make failed jobs easy to detect and rerun.
Analysis Advice (1/2)
Set up good tools for analysis.
- Compare your results to a baseline.
- Try to construct a good solution by hand before training or optimizing one.
- Use consistent themes for your plots and figures.
- Plot learning curves, success rates, failure rates, and task metrics.
- Inspect trajectories and videos.
Analysis Advice (2/2)
For papers and presentations:
- Use vector graphics over raster graphics for plots when possible.
- Generate figures for both presentations and articles.
- Show variability across seeds.
- Show representative successes and failures.
- Report the number of environment steps, evaluations, or rollouts.
- Report compute resources when they affect interpretation.
Communication Advice
Write about your work.
- Write a small bit every day.
- Write down a prediction before you run an experiment.
- Write your article outline before you run your experiments.
- Write down what would convince you that your approach failed.
- Keep a list of assumptions and known simulator artifacts.
- Document surprising failures.