Post

Simon Willison
@simonwillison.net

The reason I think it's important to spread the message that using modern AI tools is difficult is that I don't want people being embarrassed to ask questions about how to use them - or quietly assuming they "aren't smart enough" and giving up

Simon Willison@simonwillison.net

General consensus in the replies and quotes of this seems to be that the entire concept of "AI skills" is a joke - how hard is typing text into a chatbot, really? I will continue to argue that it's genuinely difficult, and that the challenge in using these tools is widely underestimated

oldest_sql_user
@exodyne.bsky.social

Oh man, I remember Reddit writhing in scorn when the term “prompt engineering” took off. The assumption was that it’s so trivially easy there was no need to overcomplicate it. Fast forward to now, when Anthropic Console will burn 60 seconds optimizing a CoT prompt for you.

Lawrence Jones
@lawrencejones.dev

To be fair, the complaint was you won’t end up with a ‘prompt engineering’ role. Which is true. Engineers are expected to learn how to engineer prompts, you don’t go hiring for someone to do this exclusively.

0 reposts1 like

mhalle.bsky.social
@mhalle.bsky.social

I think we are in the early days yet. We don't know how specialized LLMs may become. A sufficiently specialized LLM in the future may well trade features like constraints on prompting for more deterministic or correct output. The analogy today is database query engineers.

mhalle.bsky.social
@mhalle.bsky.social

It's also interesting that the current generation of LLMs benefit from good human-like communication skills, something that engineers by stereotype are not known to be especially good at (and I am an engineer).

Simon Willison
@simonwillison.net

I think prompt engineering was corrupted to mean "being good at prompting", which is clearly more a skill than a speciality But meanwhile the term "AI engineer" started to establish itself, which is the specialty of knowing how to build systems on top of this stuff - way more than just prompting

mhalle.bsky.social
@mhalle.bsky.social

I think "AI Engineer" will prove to be too broad a title to be meaningful. It could be anything from implementing neural nets in code or silicon to training, inference, or fine-tuning. It could be the AI equivalent of search, or NLP, or image processing, or decision analysis, or programming.

Simon Willison
@simonwillison.net

I disagree - a if you're training neural nets you're a machine learning engineer or a data scientist. "AI Engineer" has mostly established itself already as practitioners - using the models, not training them.

Lawrence Jones
@lawrencejones.dev

I disagree, if only because I see the engineering part of prompt went as being the process of building prompt systems not using them. The eng comes from constructing the prompts, eval’ing them, monitoring, integrating the output with other systems. Using prompts is just prompt literacy, imo.

oldest_sql_user
@exodyne.bsky.social

I like the query analogy. In theory, SQL is SQL, but "select * from employees" is a far cry from a multi-nested statement with window functions, joins and CTEs. If we have to avoid the term prompt "engineer" - and I do see the argument - we should also acknowledge that (cont)

oldest_sql_user
@exodyne.bsky.social

I've seen the complaint you reference, but I'm talking about stuff like this (mostly from 2+ yrs ago): "I think I get a little more angry every time I see someone say “prompt engineer”. [...] Like holy fuck. You are typing words to an LLM. It’s not complicated and you’re not engineering anything"

Lawrence Jones
@lawrencejones.dev

Ah right, that makes sense. I think the term makes most sense when writing prompts that power products used by others than people personally using chatbots. There is a lot of engineering required for that, it’s very difficult to do right.

oldest_sql_user
@exodyne.bsky.social

I think that's a useful distinction. I have to think about the prompts for my custom GPTs as tiny programs, but using GPT day to day is mostly just a conversation. I still do mild prompt engineering like "you're a database expert, or "include a <thoughts> tag".