viable/strict/1790175101: [Pipeline] Support outer gradient accumulation (#196641)
- PyTorch 近 90 天出现 1385 次
- PyTorch 近 90 天第 1368 次发版
- 上一次：同一天稍早 · trunk/294cb4cba0aaefd82a37854b7f470dc5c7b878cd: Support optional Tensor list schemas (#187012)
发生了什么
Add a finalize_gradients step option so callers can keep FSDP accumulation open across complete pipeline schedules. Use the public FSDP finalizer. FSDP finalization reshards parameters, but the pipeline runtime kept the stage marked as unsharded. Track stages retained across calls separately and update the runtime state when reduction reshards a stage. This preserves the duplicate-unshard checks without forcing anot…
摘要按规则整理自下方来源原文