A GPipe implementation in PyTorch
-
Updated
Jul 25, 2024 - Python
#
gpipe
Here are 8 public repositories matching this topic...
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
-
Updated
May 24, 2022 - Python
Development of Project HPGO | Hybrid Parallelism Global Orchestration
rust machine-learning tensorflow pytorch data-parallelism model-parallelism distributed-training pipedream gpipe pipeline-parallelism
-
Updated
Mar 26, 2021
A fully distributed hyperparameter optimization tool for PyTorch DNNs
-
Updated
Jan 12, 2022 - Python
Simulated Multi-GPU inference engine implementing Megatron-style Tensor Parallelism, GPipe Pipeline Parallelism, and KV Cache Sharding from scratch. Features bit-identical fidelity validation on Qwen2-0.5B weights and analytical communication cost modeling.
distributed-systems hpc distributed-computing pytorch scaling multi-gpu inference-optimization gpipe pipeline-parallelism kv-cache megatron-lm tensor-parallelism llm qwen2 torch-distributed
-
Updated
Jul 29, 2026 - Python
Add this topic to your repo
To associate your repository with the gpipe topic, visit your repo's landing page and select "manage topics."