natural_selection_2D

A sandbox for exploring natural selection with multi-agent reinforcement learning and neuroevolution in a 2D world.

What it does

• Simulates agents moving and interacting in a 2D environment
• Uses evolutionary algorithms and reinforcement learning policies
• Supports configurable agent behavior, physics, and environment rules
• Collects results suitable for experiments and model training

Project structure

• src/natural_selection_2d/ – core package
• src/natural_selection_2d/env/ – world, entities, and physics
• src/natural_selection_2d/algorithms/ – evolution logic, contains the genetic algorithm
• src/natural_selection_2d/models/ – policy definitions, includes the PyTorch AgentBrain
• src/natural_selection_2d/viz/ – rendering utilities
• scripts/ – experiment and simulation runners
• experiments/ – analysis notebooks and result artifacts

Concepts

Zero-Gradient Neuroevolution

Natural survival, agent collisions and discrete food spawns are essentially non-differentiable step functions. And standard gradient descent requires differentiable loss functions that's why Evolutionary strategies are a good fit for such usecases. This was demonstrated by OpenAI in 2017 in their paper "Evolution strategies as a scalable alternative to Reinforcement Learning" where they showed that ES can match PPO on complex continuous control benchmarks.

Neural Feedforward Policies

A simple feedforward MLP is implemented for the Agent's brain whose parameters are selected based on the fitness scores in the genetic algorithm.

Getting started

1. Install the package in editable mode:

   pip install -e .

2. Run a simulation or experiment from scripts/:

   python scripts/train_neuroevolution.py # to simulate the actual evolutionary process coupled with visuals
   python scripts/run_experiment.py # to experiment with key factors such as mutation rate, metabolism and population size

Notes

• Designed for experimentation rather than production use
• Modify config.py to tune environment and training parameters
• Use experiments/experiment_analysis.ipynb to inspect result data