Post

Totally understand the criticism but without the sweaty old coders ChatGPT would have a lot less to work with. I always thought of it as a tax haha.

0 reposts10 likes

Seems like GitHub is a better resource. Perhaps why copilot is as good as it is for vibe coding, though Claude really seems to be the stand out for me right now.

As a percentage it’s almost certainly ~0%. But I suspect it’s probably a meaningful source of much-higher-than-average-quality data.

Yeah hard to know. I would imagine a negligible impact nowadays but may have been a lot more important getting things off the ground initially. Expect it’s not something any foundational provider would be willing to share 🙈

I really need to spend some time looking at the history of synthetic data use in training. I’m vaguely aware that it’s become standard to use it but I’ve not tracked when this happened or how it was evaluated to be beneficial at all. I remember lots of fear of model collapse a year or so ago.