Tools

Simulation plays a vital role in robotics research. It is used for prototyping, testing, control design, planning, robot learning, benchmarking, sim-to-real transfer, and understanding how robots interact with complex environments.

SimR focuses primarily on rigid-body dynamics and robotics simulation, but related tools for soft-body simulation, numerical modeling, rendering, visualization, and robot description formats are also listed below.

Creating a simulation is often called modeling. In robotics, modeling is used for simulation, state estimation, prediction, control, planning, system identification, and model-based algorithms such as model predictive control and model-based reinforcement learning.

Broadly, we can list the following forms of simulation as they relate to robotics:

Often, an increase in physical accuracy comes at the cost of increased complexity, computation time, and modeling effort. For many robotics projects, the most accurate simulator is not necessarily the best simulator. The right simulator is the one that captures the phenomena relevant to the research question.

For example, a manipulation project may require careful contact modeling, while a navigation project may care more about perception, maps, and sensor noise. A locomotion project may require accurate actuator limits and ground contact, while a high-level planning project may only need approximate kinematics and collision checking.

Analytical simulations are useful when they are available, but many robotics systems do not have simple closed-form solutions. Nonlinear dynamics, contacts, friction, actuator limits, sensing uncertainty, and nonholonomic constraints often require numerical or physics-based simulation.

For a nonholonomic system, for example, one can often determine a differential relationship between state and inputs, but not a simple closed-form geometric relationship. Wheeled vehicles are a common example: knowing the total number of wheel rotations may not be enough to determine the final pose unless the full history of motion is known.

Selecting a Simulation Tool

Here are things to consider when selecting your simulation tool:

  • Research question: What do you need the simulator to tell you?
  • Robot type: Does it support your robot morphology, joints, actuators, and sensors?
  • Task type: Is the task manipulation, locomotion, navigation, aerial robotics, field robotics, multi-robot systems, or human-robot interaction?
  • Dynamics: Do you need accurate rigid-body dynamics, soft-body dynamics, fluids, terrain, or deformable objects?
  • Contact modeling: Does it handle friction, impacts, grasping, rolling, sliding, and stacking well enough for your task?
  • Sensors: Does it support cameras, depth cameras, LiDAR, IMUs, force-torque sensors, tactile sensors, GPS, or domain-specific sensors?
  • Rendering: Do you need photorealistic rendering, semantic labels, depth images, segmentation masks, or only basic visualization?
  • Robot descriptions: Does it support URDF, SDFormat, MJCF, USD, or other formats used by your workflow?
  • ROS integration: Does it integrate with ROS or ROS 2 if your project requires it?
  • Control: Can you run low-level controllers, high-level planners, reinforcement learning policies, or model predictive controllers?
  • Computation efficiency: How long does it take to run a simulation?
  • Scalability: Can you run many environments, rollouts, or random seeds in parallel?
  • Headless execution: Can you run the simulation without a GUI?
  • Determinism: Does it produce the same results given the same inputs, software versions, and hardware settings?
  • Robustness: How well does it handle edge cases, unstable contacts, extreme parameters, and long simulations?
  • Accuracy: How close is the simulation to the real system for the quantities that matter?
  • Calibration: Can you tune the simulator using real-world measurements?
  • Domain randomization: Can you randomize physical parameters, sensors, lighting, textures, objects, and initial conditions?
  • Interactivity: Can you interact with the simulation in real time?
  • Usability: How easy is it to install, configure, debug, and extend?
  • Support: Is there documentation, an active community, or commercial support?
  • Cost and licensing: Is it open-source, free for research, commercial, or license-restricted?
  • Reproducibility: Can experiments be configured, saved, replayed, and shared?

Common Robotics Simulation Use Cases

Simulation tools are often chosen differently depending on the use case.

  • Control design
    • Model predictive control
    • Trajectory optimization
    • Whole-body control
    • Legged locomotion control
    • Manipulator control
  • Planning
    • Motion planning
    • Task and motion planning
    • Collision checking
    • Navigation
    • Multi-agent planning
  • Robot learning
    • Reinforcement learning
    • Imitation learning
    • Offline policy evaluation
    • Curriculum learning
    • Sim-to-real transfer
  • Perception
    • Synthetic image generation
    • Depth simulation
    • Semantic segmentation
    • Object detection
    • SLAM and mapping
  • Design and evaluation
    • Robot morphology design
    • Actuator and sensor placement
    • Benchmarking
    • Failure analysis
    • Safety testing
  • Deployment preparation
    • Hardware-in-the-loop testing
    • Software-in-the-loop testing
    • Digital twins
    • Regression testing
    • Operator training

Robot Simulators

These tools provide higher-level robotics simulation environments, often including robot models, sensors, visualization, physics, and integration with robotics middleware.

Robot Learning Environments and Benchmarks

These environments are commonly used for robot learning, reinforcement learning, embodied AI, manipulation, locomotion, and benchmarking.

Game Engines

Game engines can be useful for robotics simulation when rendering, interaction, large environments, or custom virtual worlds are important. They are especially relevant for perception, embodied AI, human interaction, and synthetic data generation.

Physics Engines

Physics engines provide lower-level dynamics and collision simulation. Some are designed primarily for games, while others are designed for robotics, research, or scientific computing.

Game-focused or general-purpose

Research- and robotics-focused

Numerical and Scientific Computing Tools

Numerical tools are useful for modeling dynamics, solving differential equations, prototyping controllers, analyzing data, and building custom simulations.

Optimization, Control, and Planning Tools

These tools are useful for trajectory optimization, model predictive control, motion planning, and robotics algorithms.

Soft-Body, Deformable, and Specialized Simulation

These tools are useful for soft robotics, deformable objects, fluids, granular media, and specialized physical systems.

Rendering Engines

Rendering tools are useful for visualization, synthetic data, perception research, and photorealistic or non-photorealistic simulation.

Visualization Tools

Visualization tools are useful for inspecting robot models, debugging trajectories, reviewing simulation results, and communicating findings.

Robot Description and Scene Formats

Robot and scene description formats help define robot morphology, joints, inertial properties, sensors, collision geometry, visual geometry, and environments.

Authoring and Conversion Tools

These tools help create, edit, convert, or export robot models and simulation assets.

Middleware and Robotics Frameworks

These tools are not simulators by themselves, but they are often central to robotics simulation workflows.

Experiment Management and Reproducibility

Simulation-based robotics research often requires many runs, random seeds, parameter sweeps, and evaluation conditions. These tools can help with automation and reproducibility.

Physics Engine Development

These resources are useful if you want to understand or implement numerical integration, collision handling, timestepping, constraints, and other physics simulation internals.