From Tool Sprawl to Defensible Value in AI for Legal - with Christo Siebrits of AbbVie
🎯 Summary
Podcast Episode Summary: From Tool Sprawl to Defensible Value in AI for Legal - with Christo Siebrits of AbbVie
This 27-minute episode features Christo Siebrits, Senior Associate in General Counsel at AbbVie, discussing the critical challenges and strategic approaches for legal and compliance teams adopting Generative AI, particularly within highly regulated environments like pharmaceuticals. The core narrative revolves around navigating the overwhelming landscape of AI tools (“tool sprawl”), managing data governance risks, and establishing measurable value (ROI) for AI investments.
1. Focus Area
The discussion centers on Generative AI adoption within Enterprise Legal Operations, focusing specifically on:
- Data Governance and Risk Management: Balancing the use of internal vs. external (third-party) AI tools (e.g., ChatGPT, Copilot).
- Workflow Implementation: Strategies for deploying AI safely, including the necessity of “Human in the Loop” (HITL) frameworks.
- Value Measurement: The difficulty in demonstrating and defending the ROI of AI initiatives across diverse use cases.
2. Key Technical Insights
- Data Segmentation Strategy (Build vs. Buy): Organizations must segment data based on sensitivity. Highly sensitive data (e.g., R&D/discovery) necessitates building secure, internal solutions, while less sensitive enterprise querying and brainstorming can leverage buying secure, contracted third-party tools (like OpenAI with protective agreements).
- Hybrid AI Deployment: The future involves a layered approach: an internal/private LLM option for basic interrogation, integration with enterprise tools like Microsoft Copilot for file connectivity, and specialized third-party legal tools for contract navigation. No single tool is expected to solve the lawyer’s entire day.
- Evolving HITL: Human-in-the-Loop is evolving beyond simple post-check validation. It must become a “new science” of inserting human oversight at the most critical junctures (e.g., decision points, external sharing) to validate trust without defeating the efficiency gains of processing massive data volumes.
3. Business/Investment Angle
- The Cloud Analogy: The initial nervousness around sharing data with third-party AI mirrors the early adoption challenges faced with cloud computing, suggesting a necessary evolution in organizational comfort levels.
- ROI Measurement Crisis: Demonstrating defensible ROI for AI is extremely challenging. It is difficult to track productivity gains (re-deployment of time) or cost savings across disparate internal development projects versus subscription costs for third-party tools (e.g., $25/user/month across thousands of employees).
- Knowing When to Stop: A critical, underdeveloped skill is determining when to “heal off” or cancel an AI initiative because the value proposition is not being met, highlighting the need for better tracking systems.
4. Notable Companies/People
- Christo Siebrits (AbbVie): The expert providing the perspective from a highly regulated pharmaceutical environment, focusing on practical implementation and risk mitigation.
- AbbVie (ABV): The context for the discussion, representing a regulated industry where data security is paramount.
- Mentioned Tools: ChatGPT, Microsoft Copilot, Harvey, CoCounsel, LegalOn (representing the spectrum of general-purpose vs. specialized legal AI).
- Clarivate: The episode sponsor.
5. Future Implications
The industry is moving toward a nuanced, hybrid model where data residency and trust dictate the deployment strategy. The immediate future requires legal leaders to become comfortable with uncertainty regarding ROI and to actively develop sophisticated, context-specific Human-in-the-Loop protocols, especially where outputs interface with external parties (patients, partners, regulators).
6. Target Audience
This episode is highly valuable for Enterprise Legal Technology Leaders, Chief Legal Officers (CLOs), Compliance Officers, and IT/Digital Transformation Executives operating within highly regulated industries (Pharma, Finance) who are currently grappling with vendor selection, data security policies, and justifying AI spend.
🏢 Companies Mentioned
💬 Key Insights
"are we tracking the value and productivity of each tool we're deploying? And if so, how do we validate that? When do we heal off a tool? When do we cancel an initiative? When do we move on to the next thing?"
"AI is not one thing. It's different things depending on how it is deployed. For the individual, AI is ChatGPT... Then you have the developer... And then the last one is big third-party tools like Microsoft and SAP and Workday..."
"human in the loop becomes critically important when you're interfacing with an output that will be viewed or consumed by a third party."
"How do you bring all of them together? So we currently don't have a system that knits all of this together. Yes, through Copilot you can certainly knit together what's on your C drive or even what's in the cloud that's connected to you. But when it comes to actual data across the company, that's the real value of what we have..."
"It's not an AI or an AI system that gives us trust in the system. It's our ability to insert the human at the exact right juncture and then say, well, it's validated, and it's validated to the point where we have trust in it."
"So it's a question of where do you insert the human in the loop to ultimately get to a point that you feel that you can trust the data? It's not everywhere. It's just in the most critical spaces."