Post

We added a feature to Visual Studio to automatically detect when it uses an unusual amount of CPU. After 20 mins of continuous CPU across all threads combined (and the machine has been sampled), we capture a short trace.

This gets uploaded via the same pipeline that backs Windows Crash reporting. This then gets analyzed, aggregated into bug level failures, which is then ranked by hit count and impact and assigned out to teams.

It's a nice little system and resulted in a release to release decrease in CPU usage across the board for Visual Studio. It also captured some bugs that have been around forever that we would never have seen, let alone fixed without it having captured the issue.

One property I didn’t expect when we turned the feature on, was that every issue the system captured represented an actual bug or design issue with VS. I've yet to see a real legitimate use of that much CPU.

This doesn’t stop VS devs and extension authors, however, from trying to convince me that their feature is special and that they intentionally use that much CPU for their algorithm.

That is until today. I came across a resolved issue that had been marked as by design; this code legitimately spanned itself across a lot of cores with the intention of getting through this user triggered process really quickly.

And indeed, I could see that. The bursts were short, but CPU heavy that on a machine with a lot of cores but large enough dataset it could easily burn through the quota.

I went to close the bug and agree that this was actually a legitimate use case and privately lament that the "always right" of CPU tracing no longer held. But then I noticed something...