Post

Holy shit.

Why would you go there and endure the abuse of drunken sweaty old Coders when chatGPT is so respectful and correct?

Totally understand the criticism but without the sweaty old coders ChatGPT would have a lot less to work with. I always thought of it as a tax haha.

I do wonder how much SO actually factors into the training set. Feels like not very much, especially with the new synthetic data training approaches.

Yeah hard to know. I would imagine a negligible impact nowadays but may have been a lot more important getting things off the ground initially. Expect it’s not something any foundational provider would be willing to share 🙈

I really need to spend some time looking at the history of synthetic data use in training. I’m vaguely aware that it’s become standard to use it but I’ve not tracked when this happened or how it was evaluated to be beneficial at all. I remember lots of fear of model collapse a year or so ago.