iamwil

iamwil

@interjectedfuture.com

Tech Zine Issue 1: LLM System Eval https://forestfriends.tech Local-first/Reactive Programming ⁙ LLM system evals ⁙ Startup lessons ⁙ Game design quips. Longform: https://interjectedfuture.com Podcast: https://www.youtube.com/@techniumpod

2K followers368 following2.2K posts

What happened to those retro frontier models only trained on text pre-1900s? Let’s see if they can make discoveries in chemistry and discover quantum physics as a possibility. Where do they miss? Are there alternative tracks to the tech tree that we missed? x.com/emollick/st...

View full thread

Honestly, I'd been thinking about how water heaters should have an API and should be more accessible to in-home local agents. So it shouldn't have been a leap to make a joke about "someone should do this". But alas, the people actually making stuff beat me to the punch. x.com/heatbit_com...

x.comHeatbit (@heatbit_com) on XIntroducing Bitwater: the water heater element that mines Bitcoin. It screws into the electric tank you already have. Physically decentralized mining that runs no matter what 🧵 Heat water. Min…

If GPU and memory prices keep holding steady or keeps going up, finance people will start to invent financial instruments backed by hardware and trade them. But not if crypto people get there first, if they hadn't already. 😆

One crucial role of art in society is to help everyone process the emotions of change. It's therapy for an ever-changing world. Artists that long for influence and renown, but don't deeply understand the technology nor the medium are missing out on their chance to speak to all. x.com/anabology/s...

View full thread
View full thread

Not a neurologist, so an explainer helps. For me, it's the unspoken context around the original tweet thread (since I'm not the audience) that made it more difficult to understand. So having AI fill in that context for a lay person is helpful. x.com/Azadux/stat...

View full thread
View full thread
View full thread

Still getting the hang of building my own verifier. Importantly, verifiers get me out of the coding loop. When the agent can check for itself, I don't have to stay up at 2am begging it, "you fixed this, but broken that. pls fix."

I've been asking engineers if it's still important to understand the code they're shipping. The opinions are split for different reasons, and it's also a matter of degree. What's your take? Is it still important to understand what you're shipping?

View full thread

I also do a form of tactical simulation game via @ejames_c where agents and I will discuss an explainer and discuss the pros and cons of options generated by subagents for the design decision under question.

Verifiers and Guardrails: Guardrails don't change with the feature, like typecheckers, linters, and pre-commit hooks. Verifiers are different for different features, such as unit/integration tests, formal method proofs, design verifiers.

View full thread

Make the decisions that matter: Not every decision matters in a product. Agents are fast at writing code and pretty good at some of the nitty gritty details, so I let it make those decisions. I make decisions on architecture, approach, and stack that will help differentiate.

Generating explainers, quizzes, and simulations: Asking for explainers helps quite a bit, though I have a custom skill for it. Agents aren't great at it out of the box. I also have it quiz me to help retain the concepts I learned from explainers.

1. Interrogation/interview-based design and specification 2. Make the decisions that matter and leave the rest to the agent. 3. Use agent to generate explainers, quizzes, and simulations. 4. Use verifiers and guardrails in the coding loop.

Interview-based design and spec: When I tell the agent what I want to build or extend, there's naturally ambiguity. But the agent knows what is ambiguous to it, so it can ask me. Closing that specification gap is how you get better results from your agents.

View full thread
View full thread

I disagree in that you can't learn and explore the state and design space at the spec writing phase. You can't explore every nook and cranny, it's not a silver bullet. But you can learn a surprising amount you couldn't before!

Gosh, this is so awesome, I wish I thought of it. I was investigating in that neighborhood too a couple years ago, but I was too blinded by the goal of incremental computation that I missed this possibility completely. x.com/_chenglou/s...

View full thread

I also don't sync the specs. I keep an append-only log of specs, just like RFCs. How I work isn't waterfall (what Kent is thinking), nor is it what people commonly refer to now as Spec-driven. But I do agree what's important is the learning and understanding.

View full thread

So then my question: do you actually need the implementation to come to the same conclusions? Before agents, it was unequivocally yes. Now, I'm not so sure. Some things you can get away simulating with the agent, or build a prototype without the impl. What are the conditions?

Gosh, this is so awesome, I wish I thought of it. I was investigating in that neighborhood too a couple years ago, but I was too blinded by the goal of incremental computation that I missed this possibility completely. x.com/_chenglou/s...

But most awesome is the perspective shift. @_chenglou uses new engineering techniques that were simply impossible/irrelevant w/o agents, and out of anyone I follow, lives in the future of engineering. Those lamenting that engineering is over is simply just not being ambitious enough.