viable/strict/1790175101: [Pipeline] Support outer gradient accumulation (#196641)
- PyTorch: 1381 events in the last 90 days
- PyTorch: 1364th Release in the last 90 days
- Previous: earlier the same day · trunk/294cb4cba0aaefd82a37854b7f470dc5c7b878cd: Support optional Tensor list schemas (#187012)
What happened
Add a finalize_gradients step option so callers can keep FSDP accumulation open across complete pipeline schedules. Use the public FSDP finalizer. FSDP finalization reshards parameters, but the pipeline runtime kept the stage marked as unsharded. Track stages retained across calls separately and update the runtime state when reduction reshards a stage. This preserves the duplicate-unshard checks without forcing anot…
Summary assembled by rule from the sources below