Post

Tim Duffy
@timfduffy.com

I noticed that they released a version trained with QAT, but they didn't actually quantize it themselves, leaving that up to folks like lmstudio-community whoc quantized the version we tried. Any clue why they'd do it that way? huggingface.co/google/gemma...

0 reposts0 likes

Simon Willison
@simonwillison.net

No idea, I don't know why more model providers don't take that on themselves, it's pretty risky leaving it to the community as a bad quantization could give your model an undeserved bad reputation

Gus
@gusthema.bsky.social

We tried it in the past but developers like lmstudio prefer doing this themselves so giving them a better starting point seemed to work better