How Deepgram optimized voice AI at scale with AWS Trainium
- AWS: 220 events in the last 90 days
- Previous: earlier the same day · Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
What happened
Deepgram worked with the AWS Forward Deployed Engineering team to port its Nova-3 speech-to-text model to AWS Trainium2 for offline batch inference. This post covers porting to the AWS Neuron stack, kernel-level tuning, a data-parallel Rust runtime, and the throughput and cost results on batch transcription.
Summary assembled by rule from the sources below
Why it's spreading
Sources
Official
- How Deepgram optimized voice AI at scale with AWS TrainiumAWS Machine Learning