Post

whoa—underpowered compared to? this seems like a big deal

Well the metrics that are used to calibrate the thresholds (developer.chrome.com/docs/lightho...) are based off of @httparchive.org data. These servers are 2-3 times more powerful than the PSI servers so give completely different scores than the calibration date!

Case in point, I just ran it for Rothy's so I could explain the difference between field data and lighthouse. I think they got a 16. However, all CWV are green. LCP for example was 1.8s at the 75th percentile. As in, for real users. That's blowing it out of the water esp for enterprise eccommerce.

It would be a mistake to optimize the lighthouse score. The only thing that matters in web performance is user experience, which they are quite successful at

Thanks, Barry! Is it too much expensive/slow to test on packet-level throttling (like WPT's/HTTPArchive's Lighthouse)? Or to expose that as an option in APIs (maybe in some specific regions though)? I understand ++variability but ++accuracy too was sort of the conclusion many years ago

You should concentrate on your field Core Web Vitals (the top part of PageSpeed Insights), and only use the lab Lighthouse Score as a very rough guide of whether Lighthouse has recommendations to improve performance or not. 100 doesn't mean perfect, it just means Lighthouse can't help anymore. 2/5

The Lighthouse Score is best for comparing two tests made on the same Lighthouse (e.g. to test and compare fixes). Performance IS—and hence LH Scores also ARE—highly variable. LH is particularly affected by where it is run from (PSI, DevTools, CI...), but also on the lots of other factors. 3/5

Lighthouse is a GREAT tool but it also can only test some things, under certain conditions. So while it's great to see people interested in improving webperf, make sure you're doing just that (improve performance) and not just improving the score. See also: en.wikipedia.org/wiki/Goodhar... 4/5