Post

TLDR: AI progress will continue. Scaling laws, formally, may or may not, but that doesn't matter commercially. What matters is ROI: are lifecycle costs of AI worth it? That's hard to measure and predict. Even in the worst case, though, I think we're in for decades of innovation.

None of this is novel. Smarter people than me are saying the same thing. But it merits repeating as everyone (students, VCs, scientists, practitioners) nervously swings from hype to "it's all over" to "we're so back." Admire the progress. Focus on making this stuff as useful as possible

1. Of course AI is going to keep getting better. Technological progress isn't going to stop or "hit a wall." That's very safe to say. There's a ton of low-hanging fruit.

great thoughtful post @jfrankle.com. respectfully, don't agree that "bitter lesson is over". sepp hochreiter, no doubt a giant of the field, missed it. below is what another giant of the field (michael i jordan) said about deep learning for NLP in his reddit AMA back in 2015 and here we are ...

<quote start> "Although current deep learning research tends to claim to encompass NLP, I’m (1) much less convinced about the strength of the results, compared to the results in, say, vision;