trunk/6fb6ef21931fa1be2b171f5f5542730663afcf86: [FSDP2] Join pending all-gather with device stream (#199677)
- PyTorch: 1899 events in the last 90 days
- PyTorch: 1875th Release in the last 90 days
- Previous: earlier the same day · trunk/28103dea4b750fef239e4dd35d1cdc3621100751
What happened
FSDPParamGroup.finalize_backward may clean up an all-gather launched by backward prefetch but never consumed. It recorded a device-specific event and waited through torch.accelerator.current_stream(), whose generic Stream.wait_event does not handle that device-specific event correctly during CUDA graph capture. The all-gather stream remained unjoined, so capture failed with cudaErrorStreamCaptureUnjoined. Use FSDP d…
Summary assembled by rule from the sources below