Post

Curious as a potential video topic - what should developers be looking out for in the AI space in 2026?

What do you mean?

Are there particular products or subjects where you think there’s going to be a lot of progress?

Ah I see. Based on mostly vibes… - I think we’ll see 10m token context windows and substantially better long context performance so context engineering will be less important. - I think we’ll see the first usable continuous learning models towards the end of 2026, no idea what to do about that.

- I really want local models to become more of a thing but I think the RAM shortage is going to make it very hard to justify the cost of running them. They’ll continue to get better but you’ll need to use them through some inference provider for the foreseeable.

- I think we will see many more model architecture improvements compared to 2025 as the cost of scaling becomes prohibitive. The focus will be more on quality than it has been on inference cost, though inference cost will also continue to drop.

- GitHub will probably release their “agent HQ” feature that they keep talking about and try to rebuild the GitHub experience around it, with other parts of the product folding into it or languishing. There will be lots of people mad about it.

- I really want to see more companies spring up focusing specifically on agentic code review. I think current tooling is woefully insufficient. I’m thinking there’s got to be loads of great low hanging fruit in augmenting code review tools with LSP data.

These are awesome! Thanks for sharing. What do you focus on to stay up to date?

- the Dwarkesh podcast www.dwarkesh.com - everything Andrej Karpathy has ever produced (you can binge his entire YouTube channel in a couple of days) - I recently discovered semianalysis.com which covers the more business/economical side - everything sebastianraschka.com has made

- everything Welch Labs have ever made youtube.com/@welchlabsvi... - I’ve recently been making an effort to reach out and connect to folks working in ML/AI too, which has been fun. - and last but not least, Simon Willison.

Thanks so much. I realise they are pretty lazy questions for me to ask but I am genuinely very interested in the answers. And your replies are very generous ❤️