Mixture-of-Experts kernel engineering on Amazon SageMaker AI
- AWS: 206 events in the last 90 days
- Previous: earlier the same day · Announcing AWS Lambda Web Functions
What happened
AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.
Summary assembled by rule from the sources below
Why it's spreading
Sources
Official
- Mixture-of-Experts kernel engineering on Amazon SageMaker AIAWS Machine Learning