EP 524: Agentic AI Done Right - How to avoid missing out or messing up.
🎯 Summary
Podcast Summary: EP 524: Agentic AI Done Right - How to avoid missing out or messing up
This episode of the Everyday AI Show, recorded live from the IBM Think Conference, focuses on the critical need for enterprises to adopt Agentic AI correctly to maximize productivity gains while mitigating significant risks. Host Jordan Wilson interviews Dr. Maryam Asuri, Senior Director of Product Management at Watson X, IBM, to discuss the challenges and solutions for deploying AI agents at scale.
1. Focus Area
The primary focus is on the practical, production-ready implementation of Agentic AI within large enterprises. Key themes include overcoming deployment hurdles, ensuring responsible AI practices, and optimizing the cost/performance trade-off associated with scaling autonomous agents powered by Large Language Models (LLMs).
2. Key Technical Insights
- Agent Challenges Amplified: Agents inherit LLM limitations but amplify risks due to their ability to take actions, access data, interpret code, and connect to external services, making transparency and traceability of actions paramount for observability.
- Optimization via Smaller Models: The industry trend is moving toward using smaller, domain-specific LLMs to power agents. This is driven by the need to reduce the high compute costs, latency, and carbon footprint associated with larger models, especially as agentic reasoning (chain-of-thought) scales inference time.
- Lifecycle Management Focus: Successful agent deployment requires robust management across the entire lifecycle: Building, Deploying, and Monitoring. IBM’s focus is on streamlining deployment to seconds/minutes for enterprise scalability, including built-in high availability and load balancing.
3. Business/Investment Angle
- Value Realization is Key: Enterprises are still experimenting to find the “value factor” or “home moment” for agents. Success requires focusing on solving specific business problems rather than simply adopting the technology.
- Risk Sensitivity Dictates Approach: The level of human-in-the-loop intervention required depends entirely on the stakes of the use case (e.g., high-stakes finance vs. low-stakes dinner recommendations).
- LLM Acceleration to Enterprise Corners: The immediate business opportunity lies in leveraging common LLM use cases (Q&A, summarization, code generation) and then using agents to blend this acceleration into legacy systems via tool calling and workflow automation.
4. Notable Companies/People
- Dr. Maryam Asuri (IBM Watson X): Guest expert detailing IBM’s strategy for production-ready agent deployment, focusing on observability, optimization, and deployment speed.
- IBM Think Conference: The context for the discussion, highlighting recent announcements from IBM regarding Watson X capabilities.
- Watson X: IBM’s AI platform offering tools, services, and foundation models designed to help enterprises customize and deploy AI solutions.
5. Future Implications
Agentic AI is fundamentally changing how work is done, leading to significant productivity boosts. Developers are already reporting 1-2 hours of time savings daily using AI-assisted coding, freeing up time for higher-value work. The future involves a shift where non-technical users can assemble powerful, data-connected agents rapidly, accelerating the entire problem-solving process.
6. Target Audience
This episode is highly valuable for AI/ML Professionals, Enterprise Technology Leaders (CTOs, CIOs), Product Managers, and Business Leaders who are past the initial experimentation phase and are now strategizing for the secure, scalable, and responsible production deployment of autonomous AI agents.
Comprehensive Narrative Summary
The podcast addresses the urgent enterprise dilemma regarding Agentic AI: the fear of missing out versus the risk of messing up deployment. Dr. Maryam Asuri frames the discussion around moving agents from experimentation to production and scale, highlighting that the path to success is fraught with amplified challenges.
The core technical hurdles identified are Observability and Optimization. Because agents act autonomously, ensuring transparency and traceability of their actions is critical, especially in regulated industries where adherence to policy must be monitored. Furthermore, the computational demands of complex agent reasoning (chain-of-thought) drive up costs and latency. Asuri advocates for a strategic shift toward smaller, custom-tuned LLMs for agents to achieve necessary performance at a fraction of the cost.
When advising leaders on where to focus first, Asuri suggests prioritizing the Deployment phase, noting that reducing the time developers spend deploying and scaling agents from hours to minutes is a massive efficiency gain. This deployment service must include enterprise necessities like high availability and granular access control.
A key mindset shift discussed is moving away from “How can I use this agent?” to “What problem am I solving?” The stakes of the use case must dictate the level of governance required (e.g., human-in-the-loop).
Finally, Asuri offers crucial advice for avoiding mistakes: Know your limits and lines. Leaders must understand the specific risks associated with their use cases, establish clear guidelines, seek expert consultation for mitigation, and simultaneously empower their workforce by making the technology accessible rather than restricting access out of fear. The overall message is that Agentic AI is already transforming workflows, offering substantial time savings, and the key to success lies in disciplined, problem-focused implementation.
🏢 Companies Mentioned
💬 Key Insights
"Know your limits and lines. It's like what are the risks associated with your use cases that can't be jeopardized? Understanding the risks gives them a true and good lens to assess the technology."
"I run a team of product managers, and my product managers are prototyping. When we think about a new feature or idea, they are showing me the fully functional prototype that they had coded in their like, this is it, and I'm like, is it real? What am I looking at? So, I feel like this is literally changing everything. The way that we are thinking about technology, the way that we are thinking about solving problems, our problem-solving process is already changed."
"Don't go out there and try to use agents. Go out there and find a problem to solve and find the right agent that aligns with it."
"Looking to the sensitivity of the workloads. For some of the workloads, the risk is just too high that you need to make sure a human is in the loop. But for some of the use cases, like the example that I'm using is if I am using agents to provide recommendations for dinner, I probably don't care about shares of human in the loop or explainability of why I arrived at that decision."
"I would say that they should focus on the problem they are solving. Versus, hey, there is an agent, how can I use that agent?"
"So the pattern that we are seeing in the market is moving toward getting and grabbing much smaller LLMs, even for powering up agents. Why do you need unproprietary data of the enterprise, that data value users, that's their domain-specific data, to create something differentiated that delivers the performance they need for a fraction of the cost for their target use case, right?"