trunk/6b1135b1083077a8c4aa146b28f270b605e26c0a: [CI] Set NVSHMEM_MAX_TEAMS for distributed tests (#199727)
- PyTorch: 1957 events in the last 90 days
- PyTorch: 1931th Release in the last 90 days
- Previous: earlier the same day · trunk/725d0e7d7ce911862327c609493f7f862cfeabcb: Respect the blocking parameter in torch.Event (#189017)
What happened
The NVSHMEMTileCommTest tile_reduce and multi_root_tile_reduce tests (tile sizes 128 and 512) have failed on every scheduled B200 distributed run since 2026-09-09 with "No more teams available (max = 128), try setting NVSHMEM_MAX_TEAMS environment variable". Root cause: the NVSHMEM 3.7.2 bump ( #194279 ) made these tests exhaust the default team limit and worked around it by exporting NVSHMEM_MAX_TEAMS=512 in test_b…
Summary assembled by rule from the sources below