trunk/3b49aa3272e03eee4fd070851532f714995e402b: [distributed] Preserve PreMulSum in single-rank reduce-scatter (#198571)
- PyTorch 近 90 天出现 1464 次
- PyTorch 近 90 天第 1445 次发版
- 上一次：同一天稍早 · trunk/69f2e45ef3480d67dea45ec7483cab70b7c94f88: Update Metamates merge rule (#198573)
发生了什么
Human note: on NCCL2 backend this works correctly, however legacy PGNccl produces silent wrong results, so we'd better fix it. The single-rank reduce-scatter workaround copies the input to the output without applying the reduction operator, silently dropping PreMulSum's factor. NCCL 2.29.7 fixes the corruption that originally required this workaround, so use ncclReduceScatter directly on that version and newer. For…
摘要按规则整理自下方来源原文