Post

Ah I see. Based on mostly vibes… - I think we’ll see 10m token context windows and substantially better long context performance so context engineering will be less important. - I think we’ll see the first usable continuous learning models towards the end of 2026, no idea what to do about that.

- I really want local models to become more of a thing but I think the RAM shortage is going to make it very hard to justify the cost of running them. They’ll continue to get better but you’ll need to use them through some inference provider for the foreseeable.

- I think we will see many more model architecture improvements compared to 2025 as the cost of scaling becomes prohibitive. The focus will be more on quality than it has been on inference cost, though inference cost will also continue to drop.