Post

Once content is crystallized in an LLM, it is pervasive. It’s one thing for specific scientific knowledge to get updated — there’s a process around science. But implicit cultural associations? Not at all. They get built in and aren’t questioned. Also it’s unclear how you would undo them.

One big example of this is the biological, neurological, and psychological understanding of gender. The vast majority (probably 99.99%) of text that LLMs were trained on assume gender and biological sex are synonymous and binary. But science has found that they aren’t. So how would a model even get

updated? Heck, we don’t even have agreed upon language ourselves to address the distinctions (e.g. words like lesbian, gay, and bisexual all assume a binary … but binary what? Gender? Sex?) I believe that integrating AI into society will hold back our development. Hope I’m wrong.