Sam Saccone

LLM backed coding helpers suffer from the paradigm of speaking with a voice of an expert without the expertise of one. Pattern matching on well written statements alone - which is a reasonable first level triage - now is a failure mode when you bring AI to code-review.

In this way those on the receiving end of AI augmented code-review have to be skeptical out of the gate as “reasonable statements” without checking the facts leads to poor outcomes. Signal around this sentiment surfaces via some surveys survey.stackoverflow.co/2024/ai/#:~:...

“Leaked system prompts” continue to matter less and less - as proven by more and more companies open sourcing their “secret recipe” github.com/stackblitz/b... What matters is UX, interaction polish, and end-to-end value prop.

Inversely sourcegraph + cody are an example of coming up short on the UX front - a chat sidebar is not a winning approach, they have the knowledge base, the context, and the potential - but the AI UX innovation here is aged out of the gate.

“Leaked system prompts” continue to matter less and less - as proven by more and more companies open sourcing their “secret recipe” github.com/stackblitz/b... What matters is UX, interaction polish, and end-to-end value prop.

After rolling out 6 large LLM projects for developer productivity across 10k+ engineers it's clear that while the hype of "just throw an LLM at it" continues... The reality under the hood is a pile of software and a pinch of LLM.