trunk/0c8b4a78ffbbce776adc0823e790158b26435f40: flex_gemm: grouped-main SwiGLU on the grouped_mm path (#196321)
- PyTorch: 1124 events in the last 90 days
- PyTorch: 1110th Release in the last 90 days
- Previous: earlier the same day · trunk/20fb7bd04e0b24cd4299fa34d767388daa6a26a4
What happened
Human Note Agent note The dense SwiGLU spellings (interleaved acc.view(m, -1, 2) + select , and chunked split over a column-major [2N, K].t() weight) now lower on the F.grouped_mm route, so torchtitan's MoE expert FFN silu(x @ w1) * (x @ w3) over ragged experts is one grouped FlexGEMM with the same epilogue text a dense user writes. QuACK's GroupedMainStore already accepted varlen-M; only the lowering guard rejected…
Summary assembled by rule from the sources below