Post

The fact that coding agents need a lot of context to work well and chew through 10x the number of tokens (compared to regular chat) should be concrete evidence to non-coders that programmers have to hold a lot of context in their heads to be effective on production code bases.

for sure, though imo it also points to the fact that context window engineering is in its infancy. there is a lot of counterproductive accrual of redundant and irrelevant info which should prob be pruned in a granular way both proactively and by agent request

It's layered on top, for sure. But if it were efficient I don't think it would reduce the necessary context by 10x, to be on the order of chat requirements. I hear whispers of people talking about how to manage this context and memory, and it's one more curious thing I have to set to the side.

I have an itch to scratch that is the question: what would a LLM/agent-native programming language be like? So many questions to unbox from this one question.

I don’t think it’s necessary to change the embedding unless there are low level concepts that are currently far apart that should be closer together. But the hard part of programs are complexity and structure, and all the context. These are higher level concepts encoded in the higher layers of LLM.