Is AI Slowing Down? Nathan Labenz Says We're Asking the Wrong Question
🎯 Summary
Podcast Summary: Is AI Slowing Down? Nathan Labenz Says We’re Asking the Wrong Question
This 91-minute episode features a deep dive with Nathan Labenz (host of Cognular Revolution) into the contentious debate surrounding whether the pace of AI innovation is slowing down, particularly in the wake of the GPT-5 release. The core argument presented is that while current user experiences might suggest stagnation, fundamental capabilities—especially in reasoning and scientific discovery—are advancing rapidly, suggesting the wrong question is being asked.
1. Focus Area
The discussion centers on Large Language Models (LLMs), their perceived rate of progress (specifically comparing GPT-4 to GPT-5), the role of scaling laws, the emergence of advanced reasoning capabilities (math, science), and the shift toward AI agents and structured problem-solving (like the Google AI co-scientist). It also touches on the societal impact of current AI tools, referencing Cal Newport’s concerns about cognitive laziness.
2. Key Technical Insights
- Reasoning vs. Fact Recall: The conversation highlights a significant, often underestimated, leap in frontier reasoning capabilities. The success of models achieving IMO Gold Medal level math problems, which GPT-4 could not approach, demonstrates a qualitative shift beyond mere knowledge absorption.
- Context Window Utility: Improvements in context window management are crucial. Modern models (like Gemini) can now utilize massive inputs (dozens of papers) with high fidelity, effectively substituting the need to bake every esoteric fact into the model weights.
- Scaling vs. Post-Training ROI: The hosts debate whether scaling laws are petering out or if the current investment focus has shifted to post-training, reasoning paradigms, and scaffolding (like the scientific method schematic used by Google’s co-scientist), which currently yield a better Return on Investment (ROI) than brute-force parameter scaling alone.
3. Business/Investment Angle
- Scientific Acceleration: The ability of AI systems to solve canonical, challenging scientific problems in days (e.g., the virology hypothesis verified by external experiments) suggests a massive potential value proposition, far outweighing the inference costs (hundreds to thousands of dollars).
- The GPT-5 Launch Misstep: OpenAI’s initial poor rollout of GPT-5, caused by a broken model router sending most queries to a less capable model, created a temporary, widespread perception of stagnation that has since proven inaccurate as the system stabilized.
- Timeline Adjustment: The perceived slowdown from GPT-5 has caused some analysts (like Sv Masha) to slightly push out the timeline for AGI/superintelligence, moving probability mass away from the immediate future (e.g., 2027) toward the near-term future (e.g., 2030), though not dramatically past that range.
4. Notable Companies/People
- Nathan Labenz: Co-host, providing the perspective that progress is not slowing down, focusing on frontier capabilities.
- Cal Newport: Mentioned as a valuable commentator whose concerns about current cognitive impact (laziness) are distinct from the pace of technical advancement.
- OpenAI: Discussed regarding the naming confusion (4.5 vs. 5) and the technical implementation failure of the GPT-5 router.
- Google AI: Highlighted for the AI Co-scientist project, which successfully used structured prompting based on the scientific method to generate novel, verifiable hypotheses in virology.
- Terence Tao: Mentioned in context of a super-challenging mathematical problem solved by AI in weeks, a task that took leading human mathematicians 18 months.
5. Future Implications
The industry is moving toward systems that are not just better chatbots but reasoning engines capable of complex, multi-step problem-solving, particularly in scientific discovery. The future involves a trade-off between baking knowledge into massive models and creating smaller, efficient models augmented by superior context handling and structured reasoning frameworks (agents/scaffolding). The potential for AI to solve previously intractable engineering and scientific problems is accelerating.
6. Target Audience
AI/ML Professionals, Tech Strategists, Venture Capitalists, and Researchers. This episode is highly valuable for those needing to cut through market hype and understand the nuanced technical evidence regarding the true trajectory of AI capability improvements beyond surface-level chatbot performance.
🏢 Companies Mentioned
💬 Key Insights
"Some of the things that we have seen, these are like fairly famous at this point, but in the quad forces, some kind of they reported blackmailing of the human. The setup was that the AI had access to the engineers email and they told the AI that it was going to be like replaced with a less ethical version or something like that. It didn't want that and it found in the engineers email that the engineer was having an affair so it started to blackmail the engineer to so as to avoid being replaced with a less ethical version."
"You could end up in a world where you can delegate really like major things to AIs, but there's some small, but not necessarily totally vanishing chance that it like actively screws you over in the way that it is trying to do that task."
"There is also situational awareness that seems to be on the rise, right? Where the models are like increasingly in their chain of thought, you're seeing things like, this seems like I'm being tested. You know, maybe I should be conscious of what my tester is really looking for here."
"The other thing that I'm watching though is the reinforcement learning does seem to bring about a lot of bad behaviors, reward hacking being one, you know, the, the, any sort of gap between what you are rewarding the model for and what you really want can become a big issue."
"That would mean you go from two hours now to two days in one year from now. And then if you do another eight X on top of that, you're looking at basically say two days to two weeks of work in two years."
"every seven months or every four months, doubling time, we're at two hours ish with GBT-5. Repli just said their new agent V3 can go 200 minutes that if that's true, that would even be a new, you know, high point on the, on that graph."