← Back to events
ActiveAIRelease

viable/strict/1787598738: [sdpa] Fix bottom-right efficient attention when Q exceeds K (#193632)

What happened

Human Note Part 1 of doing the acutal change I want which is aligning both sdpa and varlen to lower-right the bettter choice of mask semantic in the age of llms. This fixes some edge cases that would have borken mem-eff. Currently not user facing since we are still on upper left Agent note This commit was authored with an AI assistant. CUDA memory-efficient attention stored the bottom-right causal diagonal offset as…

Summary assembled by rule from the sources below

Why it's spreading

Sources