trunk/76588348eb3e6d543e06f04751268a19f57bca8d: Fix post-local SGD checkpoint reload race (#194614)
- PyTorch: 180 events in the last 90 days
- PyTorch: 178th Release in the last 90 days
- Previous: earlier the same day · trunk/610fee8d845047496bd0713e7adff4d79d08f137: [ROCm] Fix CK SDPA flash attention on gfx1200/gfx1201 (RDNA4) (#188114)
What happened
Summary Fixes #84886 . There was a race in the post-local SGD checkpoint reload test. Rank 0 created the temporary checkpoint and all ranks synchronized after it was written. However, there was no synchronization after loading. Rank 0 could complete the test first, leave the temporary-file context, and delete the checkpoint before another rank opened it, causing FileNotFoundError. Serialize the checkpoint into an in…
Summary assembled by rule from the sources below