trunk/05b558d4e79ea7a049dbc351a9067f0bf2a63faf: [c10d] Bind local rank before eager NCCL2 initialization (#193237)
- PyTorch: 37 events in the last 90 days
- PyTorch: 37th Release in the last 90 days
- Previous: earlier the same day · trunk/26b9ddd7f8a46a15067a7cdc789623f3cdee2cb1
What happened
NCCL2 initializes eagerly, so selecting torch.cuda.current_device() before honoring LOCAL_RANK can bind forked workers to the same GPU. Prefer LOCAL_RANK and make DistributedTestBase bind the selected accelerator before init_process_group; the helper also accepts an explicit backend for compatibility tests. Test Plan: CUDA_VISIBLE_DEVICES=2,3 VIRTUAL_ENV=/home/tristanr/pytorch/.venv PATH=/home/tristanr/pytorch/.venv…
Summary assembled by rule from the sources below