SimER Tutorial

Anthony J. Clark

2024-07-26

Introduction

Format

This tutorial will follow a format in which:

We’ll get on the same page with respect to terminology
Loop
- I’ll present for a bit
- I’ll ask you all to read through an exercise
- Ask questions on our shared Google Sheet
- We’ll have time for discussion
We’ll cover a more complete example in full

About Me

I am a professor in the Compute Science Department at Pomona College in Southern California, USA.

I’ve spent arguably too much time playing around with different simulation methods (ODE, Bullet, DART, Box2D, Simulink, Chrono, MuJoCo, Gazebo, Unreal, Unity, Godot, Webots, SimPy, FEA, custom, …).

Evolutionary Robotics

The use of evolutionary algorithms to explore the design space of robots and their control software.

Research Questions

There is a wide variety of research in evolutionary robotics, but I’ll focus on:

Optimization and novelty (application-based)
- How can we optimize the design of a robot for a specific task?
- How can we explore the space of possible designs and behaviors?
Algorithm design (theory-based)
- How can we design algorithms that are more efficient and effective?
- How can we design algorithms that are more robust to noise and uncertainty?

General Algorithm

flowchart LR
    Init[initialize] --> Eval[evaluate]
    Eval --> Stop{stop?}
    Stop --Yes--> Retn[return]
    Stop --No--> Sele[select]
    Sele --> Modi[modify]
    Modi --> Evl2[evaluate]
    Evl2 --> Comb[combine]
    Comb --> Stop

population = initialize(population_size)
population = evaluate(population)

for generation in range(num_generations):
    if stop(population):
        break

    selected = select(population)
    children = modify(selected)
    children = evaluate(children)
    population = combine(population, children)

Python Implementation

Fitness = tuple[float, float]
Genome = list[float]
Individual = tuple[Genome, Fitness]
Population = list[Individual]

def initialize(size: int) -> Population: ...
def evaluate(pop: Population) -> Population: ...
def stop(pop: Population) -> bool: ...
def select(pop: Population) -> Population: ...
def modify(pop: Population) -> Population: ...
def combine(pop1: Population, pop2: Population) -> Population: ...

Simulation

This section and the related demos are the focus of this tutorial.

Types of Simulation

Analytical (closed-form) dynamics
Numerical dynamics
Rigid-body dynamics (physics engine)
Soft-body dynamics
Finite-element analysis
Digital evolution (e.g., Avida)

Simulation Advice

Choose the correct level of abstraction.

What do you care about?
Do you care about the electro-mechanical properties of the motors?
Can you get away with spherical wheels? They are faster to simulate.
Can you fake hydrodynamics or do you need full fluid dynamics?
Do you care about vision?

The main challenge: reality gap

Simulation Advice (2)

Consider your simulation characteristics.

Simulation should be noisy but deterministic.
You should evaluate a single individual with multiple initial conditions.
Your simulation should be faster than real-time.
Your simulation should be parallelizable.
Your simulation should be able to run headless.
Avoid stiff springs.
Scale your environment to the 0.1 to 10 range in terms of numerical values (1 is sweet spot).

Evolution Advice

Decouple simulation, visualization, and optimization.

Make it easy to swap out simulators.
Make it easy to swap out optimization algorithms.
Make it easy to swap out visualization tools.
You should be able to run your algorithm with different random seeds.
Be careful that your simulation environment and fitness function are not too easy to “game.”
Incorporate constraint and feasibility handling into your fitness and selection functions.
Include early stopping for your evaluations (e.g., stuck, flipped, etc.).
Include early stopping for your optimization (e.g., no progress, etc.).

Evolutionary Robotics Advice

Always prototype.

Prototype your algorithms with small populations sizes and few generations.
Test algorithms with benchmark datasets.
Test simulations with benchmark algorithms.
Test your simulation with extreme parameters.

Experiment Management Advice

Save all data if you can.

All individuals every generation.
Every replicate.
Save enough information so that you can restart your algorithm in the middle if it breaks (checkpointing).
Use a tool like wandb for tracking.

Parallelism

Consider your computational resources.

Parallelize replicate experiments (trials different random seeds)
Parallelize across generations (if mixing populations)
Parallelize across populations (multiple population per replicate)
Parallelize across individuals (multiple individuals per population)
Parallelize across evaluations (multiple evaluations per individual)
See GNU Parallel Tutorial
See Pueue is a command-line task management tool

Parallelism Advice

Plan to use \(N - 2\) cores, where \(N\) is the number of cores available
Use a tool like GNU Parallel at the replicate level
Use a library like Python multiprocessing at lower levels
Do you care more about the variance across replicates? Parallelize at the replicate level
Do you care more about the performance of a single replicate? Parallelize at the individual level

Analysis Advice

Setup good tools for analysis.

Compare your results to a baseline.
Try to construct a good solution by hand before evolving one.
Use consistent themes for your plots and figures.
Use vector graphics over raster graphics for plots.
Generate plots and figures for both presentations and articles.

Communication Advice

Write about your work.

Write a small bit every day.
Write down a prediction before you run an experiment.
Write your article outline before you run your experiments.

Exemplar

Problem Statement

Let’s start with the following problem statement:

We want to evolve an autonomous wheeled mobile robot (WMR) to navigator quickly over obstacles and then stop in front of a wall.

Online Demos

Simulation demos:

Full ER demo:

Full Example