Post

The reason I think it's important to spread the message that using modern AI tools is difficult is that I don't want people being embarrassed to ask questions about how to use them - or quietly assuming they "aren't smart enough" and giving up

Simon Willison@simonwillison.net

General consensus in the replies and quotes of this seems to be that the entire concept of "AI skills" is a joke - how hard is typing text into a chatbot, really? I will continue to argue that it's genuinely difficult, and that the challenge in using these tools is widely underestimated

15 reposts161 likes

I don't think "smart" actually has all that much to do with effective use of LLMs. Experience, sure. But I bet that good conversationalists have the most important skills: clarity, patience, curiosity, empathy. These skills translate pretty directly from human conversation to LLM chat.

Counterpoint, maybe just communicate that using modern AI is different from using a search engine? Some people will figure it out, some won't (just like search). The smarter people will ask for help. The rest will just be bad at it 🤷‍♂️

It's funny that they are called "slop machines" yet they also have a reputation for being low skill. Surely it takes skill to get something that isn't slop from the slop machine?

A lot of AI marketing does a disservice here. When you take the framing that they're a kind of computer of sorts, with its on wibbly-wobbly behaviors and specific ways you need to use it as a tool to accomplish a task, it's clear that you do need to spend time with them to be effective!

(I get so sad when I hear people dismiss their own ability to learn things, I expect that the real problem is that they were given an inaccurate mental model of the level of difficulty involved and how much work they would need to put in - I see this for learning to program all the time)

How about someone like me who uses it on a daily basis but meticulously reviews and revises the code it writes (almost the exact same process as code reviewing PRs from a new contributor) and lands them only after ensuring they have the same or higher quality as code they would write themselves?

Oh man, I remember Reddit writhing in scorn when the term “prompt engineering” took off. The assumption was that it’s so trivially easy there was no need to overcomplicate it. Fast forward to now, when Anthropic Console will burn 60 seconds optimizing a CoT prompt for you.

To be fair, the complaint was you won’t end up with a ‘prompt engineering’ role. Which is true. Engineers are expected to learn how to engineer prompts, you don’t go hiring for someone to do this exclusively.

I think we are in the early days yet. We don't know how specialized LLMs may become. A sufficiently specialized LLM in the future may well trade features like constraints on prompting for more deterministic or correct output. The analogy today is database query engineers.

It's also interesting that the current generation of LLMs benefit from good human-like communication skills, something that engineers by stereotype are not known to be especially good at (and I am an engineer).

I think prompt engineering was corrupted to mean "being good at prompting", which is clearly more a skill than a speciality But meanwhile the term "AI engineer" started to establish itself, which is the specialty of knowing how to build systems on top of this stuff - way more than just prompting

I disagree, if only because I see the engineering part of prompt went as being the process of building prompt systems not using them. The eng comes from constructing the prompts, eval’ing them, monitoring, integrating the output with other systems. Using prompts is just prompt literacy, imo.

I like the query analogy. In theory, SQL is SQL, but "select * from employees" is a far cry from a multi-nested statement with window functions, joins and CTEs. If we have to avoid the term prompt "engineer" - and I do see the argument - we should also acknowledge that (cont)

I've seen the complaint you reference, but I'm talking about stuff like this (mostly from 2+ yrs ago): "I think I get a little more angry every time I see someone say “prompt engineer”. [...] Like holy fuck. You are typing words to an LLM. It’s not complicated and you’re not engineering anything"

We’re also still discovering what tools are useful and what aren’t, what’s even possible, and the best ways to interact with them. Can’t do that without asking questions, and sharing pain points and ideas!