Post

iamwil
@interjectedfuture.com

The fact that coding agents need a lot of context to work well and chew through 10x the number of tokens (compared to regular chat) should be concrete evidence to non-coders that programmers have to hold a lot of context in their heads to be effective on production code bases.

andrew blinn
@disconcision.com

for sure, though imo it also points to the fact that context window engineering is in its infancy. there is a lot of counterproductive accrual of redundant and irrelevant info which should prob be pruned in a granular way both proactively and by agent request

0 reposts5 likes

iamwil
@interjectedfuture.com

It's layered on top, for sure. But if it were efficient I don't think it would reduce the necessary context by 10x, to be on the order of chat requirements. I hear whispers of people talking about how to manage this context and memory, and it's one more curious thing I have to set to the side.

Rasmus Andersson
@rsms.me

I have an itch to scratch that is the question: what would a LLM/agent-native programming language be like? So many questions to unbox from this one question.

andrew blinn
@disconcision.com

the lens that feels most interesting to me at the moment is as opposed to thinking about an llm-native object language, instead making each llm message a program in an llm-native metaprogramming language, making the harness a kind of metaprogramming REPL for chatting with a language server

Rasmus Andersson
@rsms.me

Have you thought about determinism? One way to think about a codebase or an evolving program is as a number of deterministic, well-defined steps. With LLMs, there’s always some “heat” and so no determinism. Ie how do you modify a program w/o affecting how most of it works

iamwil
@interjectedfuture.com

I get the sense that LLMs respond well to all the affordances that help people understand programs, due to training on human text and code. LLMs conforms to its training data. But to entertain the question, to get programming language that has affordances for LLMs,

iamwil
@interjectedfuture.com

I don’t think it’s necessary to change the embedding unless there are low level concepts that are currently far apart that should be closer together. But the hard part of programs are complexity and structure, and all the context. These are higher level concepts encoded in the higher layers of LLM.