trunk/f8d084d5b5ddddc114e3118c077e98d626a304a9: flex_gemm: support block-scaled GEMMs through QUACK (#192839)
- PyTorch: 1011 events in the last 90 days
- PyTorch: 1000th Release in the last 90 days
- Previous: earlier the same day · ciflow/trunk/197135: [UPDATE] Update
What happened
Human Note With this we can finally do full lowp mlp (dense) in SiLU -> FP8, without a BF16 intermediate rounding. return quantize_mxfp8(F.silu(acc.float())) def mlp(x, sx, w1, sw1, w2, sw2, *, tuned=True): options = {"backend": "QUACK", "tuned": tuned} hidden, sh = flex_gemm( F.scaled_mm, (x, w1.t(), sx, sw1), silu_and_quantize, gemm_kwargs=SCALED_MM, kernel_options=options, ) return flex_gemm( F.scaled_mm, (hidden…
Summary assembled by rule from the sources below