Your AI coding agent evaluation is only as good as its sandbox
- Microsoft: 57 events in the last 90 days
- Previous: earlier the same day · Microsoft announces quarterly dividend increase
What happened
Your AI coding agent passed the eval. But did the model know the answer, or did it find it somewhere on your machine? A correct answer can still invalidate your measurement.
Summary assembled by rule from the sources below
Why it's spreading
Sources
Official
- Your AI coding agent evaluation is only as good as its sandboxMicrosoft DevBlogs