trunk/3b49aa3272e03eee4fd070851532f714995e402b: [distributed] Preserve PreMulSum in single-rank reduce-scatter (#198571)
- PyTorch: 1464 events in the last 90 days
- PyTorch: 1445th Release in the last 90 days
- Previous: earlier the same day · trunk/69f2e45ef3480d67dea45ec7483cab70b7c94f88: Update Metamates merge rule (#198573)
What happened
Human note: on NCCL2 backend this works correctly, however legacy PGNccl produces silent wrong results, so we'd better fix it. The single-rank reduce-scatter workaround copies the input to the output without applying the reduction operator, silently dropping PreMulSum's factor. NCCL 2.29.7 fixes the corruption that originally required this workaround, so use ncclReduceScatter directly on that version and newer. For…
Summary assembled by rule from the sources below