viable/strict/1790933554: Add NCCL2 debug server health checks (#199092)
- PyTorch 近 90 天出现 1794 次
- PyTorch 近 90 天第 1772 次发版
- 上一次:同一天稍早 · trunk/77700655e1ddd21002e444ea9d9a501cfe259da9: [torchtitan hash update] update the pinned torchtitan hash (#199225)
发生了什么
NCCL2's flight recorder exposed a dump endpoint but did not publish watchdog health to the debug server, so the server could not trigger an all-rank dump when one rank detected a timeout or communicator failure. Port the TorchComms health-check model: have the debug frontend request NCCL2 flight-recorder dumps from every rank when any rank is unhealthy. Give a fatal abort the same configurable grace period and route…
摘要按规则整理自下方来源原文