Replying to @misaligned.markets
LLMs fail in ways people don't. The relevant question isn't whether people or LLMs make mistakes, but how mistakes are made and corrected. You cannot easily tell an LLM to correct its own mistakes, in many cases you have to fix its subsystems or data sources. Equivalent would be surgery in people?
Mike@misaligned.marketsWhen we look at LLM failure modes, they fail in exactly the ways you might predict given the shape of their corpus and training method. This is why billion-dollar companies started training models to count letters in tokenized words (how many r's in strawberry?). Failures exist in other domains too!