Post

fry69
@fry69.dev

That is simply impossible! It takes months to train a model! Fable was not available during that time where the main training run for K3 must have happened! Also it takes ENORMOUS amount of data to train a model! Way more that it is possible to siphon from an API in stealth! See also ->

Nathan Lambert@natolambert.bsky.social

How distillation is used today and what performance uplift it gives to open models (a rant) natolambert.substack.com/p/how-distil...

natolambert.substack.comHow distillation is used today and what performance uplift it gives to open modelsThis is in response to Ben Thompson’s recent piece where he said distillation is happening during RL and becoming more important to model performance.

0 reposts1 like

No replies yet.