trunk/7de327d9302fff6dc5b4271e04a7b5edadf12e33: flex_gemm: trim per-call CPU overhead in the QuACK launch path (#200712)
- PyTorch: 2301 events in the last 90 days
- PyTorch: 2271th Release in the last 90 days
- Previous: earlier the same day · trunk/10dc63912abee5b1c6b942e670872e515cbf9210: [CPP FakeTensor] Don't look up the op in register_fake (#200798)
What happened
Human Note Reduce FlexGEMM host overhead on the runtime launch path. Need for speed baby Agent note Every compiled FlexGEMM call entered torch.cuda.stream(torch.cuda.ExternalStream(stream)) and rebuilt the frozen GemmConfig from the codegen config tuple. Generated wrappers always pass the current raw stream, and QuACK's tvm-ffi launch already runs on torch's current stream, so the stream switch did nothing but cost…
Summary assembled by rule from the sources below