viable/strict/1789170654: [MPS] Migrate softmax to metal kernels (#196071)
- PyTorch: 821 events in the last 90 days
- PyTorch: 810th Release in the last 90 days
- Previous: earlier the same day · viable/strict/1789167016: [ROCm] Skip Navi3 inductor tests failing on RX 7900 (#196593)
What happened
Migrate softmax to metal kernels reusing log_softmax kernels. Perf (fp32 mainly in benchmarks because Transformers standard sampling path casts logits to fp32 before softmax): Shape Dtype Dim Layout MPSGraph (us) Metal (us) Speedup 1x128256 fp32 -1 contiguous 57.37 7.14 8.035x 1x129280 fp32 -1 contiguous 50.05 7.09 7.060x 1x151936 fp32 -1 contiguous 56.22 7.58 7.420x 1x154880 fp32 -1 contiguous 60.75 8.02 7.576x 1x2…
Summary assembled by rule from the sources below