AGI is still 30 years away — Ege Erdil & Tamay Besiroglu
100
Companies
301
Key Quotes
5
Topics
8
Insights
🎯 Summary
Podcast Summary: AGI is Still 30 Years Away — Ege Erdil & Tamay Besiroglu
This 188-minute episode features Ege Erdil and Tamay Besiroglu, founders of the new company Mechanize (focused on work automation), discussing their contrarian, longer-term view on achieving Artificial General Intelligence (AGI) compared to the prevailing Silicon Valley sentiment.
1. Focus Area
The discussion centers on Artificial General Intelligence (AGI) timelines, the nature of technological acceleration, the limitations of current AI capabilities, and the necessary prerequisites for achieving broad economic transformation. They contrast the concept of an “intelligence explosion” with a more complex, multi-faceted technological revolution akin to the Industrial Revolution.
2. Key Technical Insights
- The “Intelligence Explosion” Analogy is Misleading: The hosts argue that focusing solely on an “intelligence explosion” is like focusing only on “horsepower” during the Industrial Revolution. True transformation requires numerous complementary innovations across various sectors (finance, law, infrastructure), not just one core capability leap.
- Capability Unlocks vs. Compute Scaling: Progress in AI appears rapid because major capabilities (like sophisticated reasoning or coding) are unlocked periodically, often coinciding with large steps in compute scaling (e.g., 9-10 orders of magnitude since AlexNet). However, they suggest the remaining necessary unlocks (e.g., long-horizon agency, full multi-modality) might require significantly more compute scaling than the economy can easily sustain, or they might require fundamentally new innovations beyond current scaling laws.
- The “Unhobbling” Hypothesis vs. New Capabilities: They debate whether current models are “baby AGIs” that just need “unhobbling” (better context, agency scaffolding, post-training) or if entirely new, difficult capabilities must be engineered. They lean toward the latter, noting that while LLMs excel at knowledge retrieval (like answering Pokemon questions), they fail at executing complex, novel tasks within a dynamic environment (like playing an unknown Steam game).
3. Business/Investment Angle
- Longer Timelines for Full Automation: Tamay predicts a drop-in remote worker replacement around 2045, while Ege is slightly more bullish but still suggests timelines extending significantly beyond the common 2027-2030 predictions. This implies that the immediate economic disruption might be slower than anticipated by hyper-bulls.
- Job Automation Nuance: Many current tasks people perform are only a small fraction of their overall job. Automating that single task (e.g., booking a flight) does not automate the entire job, suggesting the fraction of the economy automated by AI is currently very small.
- Revenue as a Proxy for Utility (Debated): The discussion touches on whether massive revenue (e.g., OpenAI hitting $100B) is evidence of transformative intelligence. They caution that people pay trillions for non-transformative things (like oil), suggesting high revenue alone isn’t definitive proof of AGI readiness.
4. Notable Companies/People
- Ege Erdil & Tamay Besiroglu: Founders of Mechanize, advocating for longer AGI timelines and a broader view of technological change.
- Robin Hanson: Mentioned as an example of someone whose extrapolation of current automation trends suggests centuries until full automation.
- OpenAI: Referenced regarding their potential $100B revenue milestone, which they see as a weak signal for AGI progress unless the figure is significantly higher (e.g., $500B).
- WorkOS: Mentioned in an ad break as a company helping software firms transition from consumer-grade products to enterprise-ready solutions (SSO, audit logs), highlighting the difficulty of building necessary infrastructure features.
5. Future Implications
The conversation suggests that the path to AGI is not a straight extrapolation of current LLM performance. Significant, hard-to-predict breakthroughs in areas like long-horizon agency, robust multi-modal integration, and general environmental interaction (not just text-based reasoning) are still required. If these breakthroughs are as difficult as scaling compute constraints suggest, AGI remains decades away.
6. Target Audience
This episode is highly valuable for AI researchers, venture capitalists, technology strategists, and technical professionals interested in the fundamental drivers and realistic constraints shaping the AGI timeline, moving beyond hype cycles.
🏢 Companies Mentioned
Apple
✅
big_tech
Elon Musk
✅
individual_reference
Elias Hatske
✅
individual_reference
Jeff Dean
✅
individual_reference
SpaceX
✅
organization_example
AI platforms
✅
ai_application
John Monnoiman
✅
hypothetical_example
Matthew
✅
individual_researcher
Scott
✅
individual_researcher
Daniel
✅
individual_researcher
Metaclis
✅
organization
O3 minis
✅
ai_application
Robin Hanselmixer
✅
ai_commentator
AI I
✅
unknown
China I
✅
unknown
💬 Key Insights
"for humans you like every human has to learn things from scratch basically like they are born and then they have a certain and a lifetime learning that they have to do so in human learning there is a ton of duplication well for an AI system it could just learn once you just have one huge train run which a tons of data and then that run could be deployed everywhere"
"the fact that I can like this is the skill I need or the set of skills I need and I can have a worker and just like I can have a thousand workers in parallel if there's something that has a high elasticity of demand I think is like probably along with the transformative AI the most underrated tangible thing that like you need to understand about what the future AI society will look like"
"firms right now have two of the three relevant criteria for evolution they have selection and they have variation but they don't have high fidelity replication and you could imagine a much more fast-paced and intense sequence of evolution for firms once you once you have this final piece click in"
"the crucial point we were making was that people tend to overemphasize and think of AI from the perspective of how smart individual copies will be and if you actually want to understand the ways in which they are superhuman you want to focus on their collective advantages which because of biology we are just precluded from"
"there's an argument about growth levels so we're saying we're gonna see 30% growth per year instead of 3% they responded at with an objection about levels so they say well how much more efficient how much more valuable can you make like hairdressing or like taking flights or whatever or going to a restaurant and like that is just fundamentally the wrong kind of objection"
"we can think about you know an h100 does about there are some estimates of how much computation the human brain does per second and it's about one e15 flopper so it's a bit a bit unclear but and then it turns out that an h100 roughly does on that order of computation so you can ask the question of how long does it take for an h100 to pay itself back if you run the software of the human brain if you run the software of the human brain you can then deploy that in the economy and earn say human wages on the order of 50 to 100k a year or whatever in the US and so then it pays itself back because it costs on the order of 30k per h100 and so you get a doubling time of maybe on the order of a year"
📊 Topics
#artificialintelligence
429
#aiinfrastructure
32
#generativeai
7
#investment
4
#startup
3
🧠 Key Takeaways
expect that AI systems are able to make a lot more progress on that
recognize that this is a very narrow subset of relevant tasks that humans do in order to be a competent, economically valuable agent
do it this way
just start our own org because we can just hire people and work on the projects we were excited about and then I just you know hired a bunch of the insightful misfits that like but did you like with was it with the thesis like oh there's a bunch of underutilized internet misfits and therefore like this org was just as for you started the organ then you're like I think it's more of the latter so it was more like we could make a bunch of progress because clearly like academia and industry is kind of dropping the ball on a bunch of important questions that academia is is unable to publish interesting papers on industry is not really focused on yeah producing useful insights and and so it seemed like very good for us to just do that and also the timing was very good so we started just before you know chat GPT and we wanted to have much more grounded discussions of the future of AI yeah and I was frustrated with the quality of discussion that was was happening in on the on the internet about the future of AI and I mean to some extent or to a very large extent I still am yeah and that that's like a large part of what you know motivates me to do this is just like born out of frustration with bad thinking and arguments about where AI is going to go the part about my job that I enjoy the least is the post production I have to rewatch the episode multiple times make all these difficult judgment calls and I've been trying to automate all this work with LLM scripts and I found that Google's Gemini 2
have a similar attitude towards other industries that it's like much more complicated right I mean it's so Robin Hansen has this abstraction of like seeing things in near mode or just farm mode right and I think if you don't know a lot about the topic because then you see it sort of in farm mode and you sort of simplify and that's right things you know you see a lot more detail like in general I think the thing I would say and the reason I also believe that just like abstract reasoning and like sort of structure reasoning or even Bayesian reasoning by itself is not like sufficient or like it's not as powerful as many other people think is because I think there's just this like enormous amount of like richness and detail in the real world that like you just can't like reason about it right you you need to see it and obviously that like that is not an obstacle to AI being incredibly transformative because as I said like you can scale your data collection you can scale experiments you do both in the AI industry itself and just more broadly in the economy so you just discover more things more economic activity means we have more exposed surface area to have more discoveries like all of these are things that have happened in our past right there's no reason that they they couldn't speed up like the like the fundamental thing is that there's no reason fundamentally why economic growth can't be much faster than it is today like it's probably about us to us right now just because humans are such an important bottleneck they both supply the labor they play crucial roles in the process of like discovery of various kinds of productivity growth there's just strong complementarity which doesn't make sense with capital that you can't substitute machines and so on for humans very well so the growth of the economy and growth growth productivity just ends up being bottlenecks by the growth of human population publicly available data is running out so major AI labs like meta google deep mind and open AI partner with scale to push the boundaries of what's possible through scales data foundry major labs get access to high quality data to fuel post training including advanced reasoning capabilities as AI research forward we must also strengthen human sovereignty scales research team seal provides practical AI safety frameworks and validates frontier AI system safety via public leaderboards and creates foundations for integrating advanced AI into society most recently in collaboration with the center for AI safety scale published humanities last exam a groundbreaking new AI benchmark evaluating expert level reasoning and knowledge capabilities across a wider range of fields if you're an AI researcher or engineer and you want to learn more about how scales data foundry and research lab can help you go beyond the current frontier of capabilities go to scale dot com slash the war cash let me ask this in general question what is what has happened in China over the last 50 years yeah would you describe that as like in principle the same kind of explosive growth that you expect from me because there's like a lot of labor that makes the marginal product of capital really high which allows you to have like 10 percent plus economic growth rates is that basically in principle for me I so I would say in some ways it's similar in some ways it's not the way probably the most important way which is not similar is that in China you see it is relative like you see a massive amount of capital accumulation yeah substantial amount of adoption of new technologies and probably also human capital accumulation to some extent and but you're not seeing a huge scale up in the labor force in the fake labor force while for AI you should expect to see a scale up in a labor force as well not in human workforce but in the AI workforce I think you did kind of like well maybe not consecutive increases in the labor forcing but like you did the key thing here is just the simultaneous scaling of both these things and so like you might ask the question of isn't it like basically half of what's going to happen with AI that you scale up you know capital accumulation in China but actually that's really not like if you get both of these things to scale that that gives you just much faster growth and a very different picture but at the same time if you're just asking like what would 30 percent growth per year like look like like in terms of like if you're just gonna have an intuition for how transformative that would be in concrete terms then I think looking at Chinese not such a bad case like you can especially in the 2000s or maybe late 90s like that gives you a good that seems slower than over forecast right I think also looking at the industrial revolution is pretty good when that's revolution is very slow so but just in terms of the types of the kind of the margins along which we made progress in terms of products so what what didn't happen you the thing that didn't happen during the industrial revolution is we just produced a lot more of things that people were producing prior to the industrial like producing a lot more crops and maybe a lot more kind of pre-industrial revolution style houses or whatever on farms instead what we got is along pretty much every main sector of the economy we just had many different products that are totally different from what was being consumed prior to that so in transportation in food in healthcare is a very big deal and to be out of so another question because I'm not sure I understand like how you're defining the learning by doing versus explicit R&D because there's like the way for taxes that companies say what they call R&D but then there's like the intuitive understanding of R&D so if you think about how AI is boosting a TFP you could say that like right now if you just had replaced the TSMC process engineers with AI's and they're finding different ways in which to improve that process and like improve efficiencies improve yield right I would kind of call that R&D on the other hand you emphasize this the other part of TFP which is like better management then learning by doing that kind of stuff but learning by doing could be you could I mean how much how much on for you like you're gonna get to the you're gonna get to like the fucking Dyson spear by better management like it's not just but but but that's not the argument right like the point is that there are all these different things that like somewhat that might be more complimentary than others the point is not that you can get to a Dyson Sphere by just scaling labor and capital like that's not a point like you need to scale everything at once so just as you can't get to a Dyson Sphere by just scaling labor and capital you also can't get to it by just scaling TFP that doesn't work I think there's a very important distinction between what is necessary you know to scale to get the this you know Dyson Sphere world and what is important like in some sense producing food is necessary um but but of course producing food doesn't get you to Dyson Sphere right so I think R&D is necessary but on its own isn't sufficient and scaling up the economy is also necessary on its own is not sufficient and then you can ask the question what is the relative importance of each yeah so I think our view here is like very much the same we're like like it is very connected to our view about the software and do you think we were just saying like there are these bottlenecks so you need to scale everything at once like this is just a general view but I think people like misunderstand us sometimes as saying that uh like R&D is not important like no that's not that's not what we what we're saying we're saying it is important it is less important in relative terms than some other things none of which are by themselves sufficient to enable this growth so the question is like how do you do the credit attribution I mean one way in which in economics standard do that is to look at the elasticity of output to the different factors like like capital is less important than labor because the output elasticity of like labor elasticity output is like 0