viable/strict/1791282642: [FSDP] Reduce TestParityWithDDP GPU delays 250ms -> 50ms (#199769)
- PyTorch 近 90 天出现 1964 次
- PyTorch 近 90 天第 1938 次发版
- 上一次：同一天稍早 · trunk/381cc3e476bad355609f87c1634e843f8d335df0: [audio hash update] update the pinned audio hash (#199869)
发生了什么
TestParityWithDDP's delay tests ( test_delayed_optim_step , test_delayed_reduce_scatter , test_mixture_of_experts_with_delay_before_free ) inject a 250ms torch.cuda._sleep per iteration so the GPU lags the CPU and missing FSDP stream syncs show up as parity mismatches. About 410s of the class's ~917s per CI config is these sleeps. TestParityWithDDPCUDA is the largest class in test_fsdp_core (~424 aggregate CI minute…
摘要按规则整理自下方来源原文