Post

Machine learning is easily the most amazing technology I've witnessed in 50 years. AI is advancing so fast, and I'm not 100% sure where the public pulse is on this one. Has it swung fully into raw hatred? There's good reason for that. But it's also here, and getting astonishingly good.

0 reposts6 likes

Agentic AI like Cursor or Cline, models like Claude 3.7 are just an absolute paradigm shift for engineering. I can't even put into words how much of a seismic shift the last few months have been. Engineering loves this stuff, the art and media world, not so much. Those new video models tho, wow.

I only subscribe to ChatGPT Plus, but I've watched videos about AI-connected IDEs, and I feel like I'm missing out. It's been so incredibly helpful just as a chatbot. How good is the context window? How does it handle large code bases?

It uses a treemap to get an idea of all function signatures in the code base, then uses function calling to instruct the IDE to open/edit files, run shell commands and evaluate their output etc. Ive used it on giant code bases, green field, UI dev. It's a game changer compared to just using ChatGPT

Context sizes have increased dramatically, as have context management techniques, to the degree that I don't really think much about context sizes anymore. In terms of models, Claude 3.7 is the pinnacle of coding models right now, and in terms of agents: Cline, Aider, Claude Code.

"GenAI" is a marketing concept that pickbacks on the great advances of so many years of ML, despite being functionally the opposite (using patterns to generate content rather than deriving patterns from data) to sell a fantasy of the machine "thinking".

They're technical rooted in some of the same fundamentals, but those who profit from selling "AI" have purposely conflated them to justify products that are still probability algorithms pretending to do thoughtful or creative work.

I worry that LLMs/image GAN have set back the actual AI field (mostly ML) as the image of ML is likely now tainted by the false promises of some sentient super-being computer that has never and will never exist - a myth propped up to sell GPUs.

That's a nonsensical distinction. ML is what creates gen AI models. You still derive patterns from massive amounts of data when you train e.g. an LLM. That those models can then in turn be used to create data doesn't change the fact that both are firmly rooted in ML.

I love your description of AI as the opposite of ML (the garbage collector has become the garbage producer) but I think that might be a natural evolution of the technology. It will be fun to see where science fiction and reality collide. People are willingly fooled all the time.

It's a very promising tool that can be used by professionals to cut down a lot of tedious busywork - unfortunately, those in power are pushing it as a tool to cut down on professionals; to replace skilled work with auto-generated slop to maximise profit by minimising the expense of skilled creators.

Absolutely. And for the time being, since it's auto-generated slop, it doesn't really resonate either. But I wonder how long until it cuts through; what defenses we'll need to develop as consumers, and how the most resilient human jobs will incorporate it into their work in impactful ways.

I'm not convinced that you will ever get something truly meaningful without the guiding hand of a skilled creative - the AI can't think or feel; it can't have original ideas; it can only regurgitate what it has been fed. I worry, however, that that low-quality slop will be pushed for normalisation.

There is also the problem that it is running out of data to be trained on. They've had to delay rolling out new model versions because it's hitting a wall. Once GenAI reaches its limit on being able to steal original works it will have to cannibalize its own slop creating a feedback loop of trash.

I agree -- not convinced of that either. But I also think there's a world of utility that either doesn't require those traits, or an approximation is good enough. It's profoundly useful for programming, for instance, or learning new software tools/languages.