Description
Have a pool of buffers for receivers, when an actor wants to receive a tensor is gets a preregistered buffer from the pool and receives into it, when time to release comes- return the buffer to the pool
Use case
When using rdt/nixl in the receive side registration can be time consuming, there is an option to do set_target_for_ref but that limits us to get exactly the same shape of tensor each iteration. have a memory pool like in the send side can save registration time and improve performance.
Description
Have a pool of buffers for receivers, when an actor wants to receive a tensor is gets a preregistered buffer from the pool and receives into it, when time to release comes- return the buffer to the pool
Use case
When using rdt/nixl in the receive side registration can be time consuming, there is an option to do set_target_for_ref but that limits us to get exactly the same shape of tensor each iteration. have a memory pool like in the send side can save registration time and improve performance.