Take a look at [PPO implementation details, bullet #1](https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/). There even a support in ```pytorch``` for multiprocessing.
Take a look at PPO implementation details, bullet #1.
There even a support in
pytorchfor multiprocessing.