Post

Very proud to share this new work led by the amazing Marlou Rasenberg (not on here), with Joel Fischer and me doi.org/10.5281/zeno... We use #EMCA methods to study empirically how LLM users shoulder the burden of solving sequential trouble and end up restructuring their own aims & activities

We identify a set of practices we call 'pre-emptive prompting', "through which users creatively circumvent the system’s lack of action recognition and action projection". Users stumble upon these through trial and error, in exchanges like this:

While much work in so-called 'prompt engineering' obsesses over the perfect prompt (Andreessen's monster prompt comes to mind), we find that real-life prompting practices are sequentially organized, feature repurposed patterns of human interaction, and result in users adjusting expectations & aims

As we conclude, > [LLM] users encounter a fundamental asymmetry: they must singlehandedly supply much of the sequential structure that is ordinarily co-constructed by multiple participants ... This hidden labour challenges accounts of LLM use that focus on performance and optimization

I can just see no logical inference except that a tool that mimics thought in language without being it will always and inevitably increase/generate workload for any mind participating, and the more convincing the fakery the greater the workload increase.

I think so. It has reasoning. But also in the system as a whole, we respond automatically to beings with minds, so it's always intoducing a massive source of error that, at best, we expend great cognive resource and effort to parse/resist/put aside. Just can't see how that could ever aid efficiency.

Correction: it doesn't have reasoning, though it is cleverly finetuned to give that impression. As reviewed here: doi.org/10.5281/zeno... But yes, since we are used to read between the lines, we overattribute intentions and meanings to synthetic text (as Emily Bender often points out)

The size of the error and the effort increasing with the degree to which our minds sense 'being with mind communicating'. Considering too this is survival critical neuropsychology so ultra rapid, entirely outside awareness, built to take over all internal and behavioural responses of human system?

Or if not survival critical, at least crucial in human interaction? Elsewhere I described these 'lopsided sensemaking' processes, and speculated that the highly skewed degree of investment makes it much harder for people to let go of the 'distributed delusions' (Osler) doi.org/10.5281/zeno...