Post

Very proud to share this new work led by the amazing Marlou Rasenberg (not on here), with Joel Fischer and me doi.org/10.5281/zeno... We use #EMCA methods to study empirically how LLM users shoulder the burden of solving sequential trouble and end up restructuring their own aims & activities

We identify a set of practices we call 'pre-emptive prompting', "through which users creatively circumvent the system’s lack of action recognition and action projection". Users stumble upon these through trial and error, in exchanges like this:

While much work in so-called 'prompt engineering' obsesses over the perfect prompt (Andreessen's monster prompt comes to mind), we find that real-life prompting practices are sequentially organized, feature repurposed patterns of human interaction, and result in users adjusting expectations & aims

As we conclude, > [LLM] users encounter a fundamental asymmetry: they must singlehandedly supply much of the sequential structure that is ordinarily co-constructed by multiple participants ... This hidden labour challenges accounts of LLM use that focus on performance and optimization

Super interesting (and appreciate as always that it's accessible to someone like me)! It seems like hidden labor is not uncommon in sw products, but ime because it's better masked in llms, coworkers can fail to realize they should have kept laboring a lot longer before sharing their output.