Post

The fact that coding agents need a lot of context to work well and chew through 10x the number of tokens (compared to regular chat) should be concrete evidence to non-coders that programmers have to hold a lot of context in their heads to be effective on production code bases.

for sure, though imo it also points to the fact that context window engineering is in its infancy. there is a lot of counterproductive accrual of redundant and irrelevant info which should prob be pruned in a granular way both proactively and by agent request

I have an itch to scratch that is the question: what would a LLM/agent-native programming language be like? So many questions to unbox from this one question.

I’ve had the same itch after experiencing just how *different* it is to work with agents on Rust vs JavaScript.

the lens that feels most interesting to me at the moment is as opposed to thinking about an llm-native object language, instead making each llm message a program in an llm-native metaprogramming language, making the harness a kind of metaprogramming REPL for chatting with a language server

I get the sense that LLMs respond well to all the affordances that help people understand programs, due to training on human text and code. LLMs conforms to its training data. But to entertain the question, to get programming language that has affordances for LLMs,

I don’t think it’s necessary to change the embedding unless there are low level concepts that are currently far apart that should be closer together. But the hard part of programs are complexity and structure, and all the context. These are higher level concepts encoded in the higher layers of LLM.