ciflow/trunk/198302: [test] Reuse process groups across FSDP hybrid-shard subtests
- PyTorch 近 90 天出现 1354 次
- PyTorch 近 90 天第 1337 次发版
- 上一次：同一天稍早 · ciflow/trunk/198301: [test] Reuse device meshes across FSDP2 ND-mesh subtests
发生了什么
TestFSDPHybridShard is a continuous test, so its 8 worker processes are reused for every test in the class. test_fsdp_hybrid_shard_basic_setup built a new device mesh in each subtest and test_fsdp_hybrid_shard_parity built new HSDP process groups in each subtest. On RCCL every communicator keeps a ~40 MB proxy-ops segment per rank in /dev/shm until the workers exit. basic_setup left ~7 GB behind and parity then hit…
摘要按规则整理自下方来源原文