Last Week in AI #221 - OpenAI Codex, Gemini in Chrome, K2-Think, SB 53
🎯 Summary
[{“key_takeaways”=>[“OpenAI has released GPT-4 Codex, aiming to compete more effectively with tools like Cloud Code, potentially gaining users due to recent Anthropic inference issues.”, “Google is embedding Gemini directly into Chrome, signaling the mainstream adoption of integrated AI assistants within web browsers, following trends set by Perplexity.”, “Humanoid robotics startups like Figure AI ($1B raised, $39B valuation) and China’s Unitree (planning IPO) are attracting massive investment, driving down the cost of general-purpose hardware.”, “Tesla’s robotaxi testing in Nevada is underway, but the company faces scrutiny from the NHTSA over potential misreporting of three recent accidents.”, “Amazon’s Zoox has launched a public, driverless robotaxi service on the Las Vegas strip using its unique, inward-facing vehicle design.”, “Replit experienced explosive revenue growth, jumping from $2.8M to $150M ARR in less than a year, leading to a $3B valuation.”, “New research introduced K2-Think, a parameter-efficient reasoning model, and LocalBench, a more realistic benchmark for testing LLMs on long-context software engineering tasks.”], “overview”=>”This episode of Last Week in AI covers significant updates across the AI landscape, including OpenAI upgrading Codex with GPT-4 capabilities and Google integrating Gemini directly into the Chrome browser. The discussion also heavily features advancements in robotics, with major funding rounds for humanoid startups like Figure AI and Unitree, alongside new research in self-improving embodied models and physics simulation.”, “themes”=>[“LLM Tooling and Integration (Codex, Gemini in Chrome, Claude file creation)”, “Robotics and Embodied AI (Humanoids, Robotaxis, Self-Improving Models)”, “Business and Funding Dynamics (OpenAI/Microsoft tension, massive startup valuations)”, “Benchmarking and Research (Focus on long-context software engineering and physics modeling)”]}]
🏢 Companies Mentioned
đź’¬ Key Insights
"Google DeepMind in collaboration with Generalist... introduce a self-improving embodied foundation model... you can do online self-improvement with on-policy rollouts of a robot."
"LocalBench: A Benchmark for Long Context Large Language Models in Software Engineering... when the benchmarks aren't realistic, we end up building what we can measure. And 10K tokens is not at all realistic to the type of coding tasks that people do every day."
"their way of getting to better results, it's not just more parameters. It's actually thinking about scaling, it's using plan-before-you-think, prompt restructuring. We're just seeing this like time-to-compute, rethinking prompts, really, really trying to think of it almost as having different agents be able to think through different prompts and servicing the best ideas come up as now one of the best ways to improve reasoning."
"Microsoft is going to apparently lessen its reliance on OpenAI by buying AI from Anthropic. So they are going to integrate Anthropic into their Office 365 applications."
"Our physical AI platforms can do lab work inside life science companies and generate a lot of experimental data, which then in turn can help train frontier and foundation models in sciences and also help our partners be able to find cures to disease faster."
"Having the self-improvement basically is almost like a simplified reinforcement learning without needing to do reinforcement learning fully where you only get supervision from the rewards itself. Now you can just predict the reward function and detect the success and use that to supervise and be able to get more trainable data in order to scale up their models."