Post

there’s a joke to be made here about doing it for the golden ratio, I can sense it but it’s out of reach

i would think that going from larger to smaller you wouldn’t even need NVIDIA’s thing (which was lossy btw)

hmm you'd probably know better than i would, but i believe the other paper was doing this roughly to enable cheaper models to be used for tasks by latger supervising models