How Financial Services Are Building Safer Customer-Facing AI - with Akhil Khunger of Barclays
🎯 Summary
Comprehensive Summary: How Financial Services Are Building Safer Customer-Facing AI - with Akhil Khunger of Barclays
This podcast episode, featuring Akhil Khunger, VP of Quantitative Analytics at Barclays, focuses on the critical challenge of embedding safety, security, and robust governance into customer-facing Artificial Intelligence (AI) systems within the highly regulated financial services industry. The discussion moves beyond backend stress testing to address the specific risks associated with deploying generative AI directly to clients, emphasizing the need to balance user experience with regulatory compliance and risk mitigation.
1. Focus Area
The primary focus is Customer-Facing AI Safety and Governance in Financial Services. Key topics covered include mitigating specific AI vulnerabilities (prompt injection, jailbreaking), the role of model governance and validation teams, the impact of global regulatory shifts (like the EU AI Act), and the cautious future deployment of agentic and generative AI in client interactions.
2. Key Technical Insights
- Layered Input Protection: Safety begins before the model sees the input; governance teams implement layers to restrict access to certain AI features or data, effectively filtering out potential threats or inappropriate data flows before they reach the core model.
- Comprehensive Test Cases in Production: Financial institutions must utilize old-school, comprehensive test cases that are continuously run, even in production environments, to catch anomalies arising from new data or unexpected user prompts, with failures triggering immediate alerts.
- Decoupling Agentic Processes: For emerging agentic AI, understanding the model requires decoupling the complex, multi-layered processes to test individual components for correct output, rather than relying solely on the final result of the autonomous chain.
3. Business/Investment Angle
- Regulatory Compliance as a Driver: Regulatory pressure necessitates robust documentation, evidence of testing, and transparency to regulators, which can sometimes require sacrificing certain functionalities that might otherwise enhance the user experience.
- High Investment in Testing for Agentic AI: The adoption of sophisticated agentic systems will require significant upfront investment in testing infrastructure and human oversight (benchmarking against human performance) before widespread deployment, potentially slowing adoption rates compared to less regulated sectors.
- Balancing UX and Security: For high-value clients, maintaining a “velvet rope” experience means integrating AI support with dedicated relationship managers to guide prompt usage and ensure comfort, though this level of personalized oversight is not scalable for the general customer base.
4. Notable Companies/People
- Akhil Khunger (Barclays): The featured expert, providing practical insights from a major financial institution on AI governance and risk management.
- Barclays: The context for the discussion, representing a large, regulated entity navigating these challenges.
- OpenAI (Mentioned): Referenced as an example of a foundational model provider whose outputs require significant pre-processing and governance layers by the bank.
5. Future Implications
The industry is moving toward a future where trust and transparency are visible operational requirements, not just backend processes. While agentic AI holds immense potential for automation, its deployment in customer-facing roles will be significantly slower and more deliberate in finance due to the complexity of auditing multi-step autonomous actions and the stringent regulatory environment. There will be a greater emphasis on using AI itself to analyze user interactions (e.g., time spent on a task, repeated errors) to contextualize feedback scores and identify genuine issues versus false positives.
6. Target Audience
This episode is most valuable for AI/Tech Professionals, Risk Management Leaders, Compliance Officers, and Business Strategists operating within highly regulated industries, particularly Financial Services, who are responsible for deploying, governing, or investing in customer-facing generative AI solutions.
Comprehensive Narrative Summary:
The conversation with Akhil Khunger centered on the transition of AI safety from internal stress testing to external, customer-facing deployment. Khunger stressed that mitigating risks like prompt injection starts at the input stage, requiring governance teams to strip down foundational models before use. A core defense mechanism involves rigorous, comprehensive test cases that must function even in live production, validated by dedicated model governance and validation teams before any system goes live.
A major concern highlighted is the integrity of the training data, which may contain information irrelevant or even prohibited for financial use cases, necessitating careful curation. Furthermore, Khunger addressed the delicate balance between security and user experience (UX). While high-value clients may receive personalized human oversight to ease them into new AI tools, scalability demands that the UI itself be rigorously tested for experience, not just accuracy, with feedback loops informed by AI analysis of interaction metrics.
The discussion then pivoted to regulatory pressures, particularly the EU AI Act, noting that compliance often forces trade-offs against optimal UX, requiring banks to meticulously document all testing and governance processes for regulators demanding transparency. Finally, the future of agentic AI was examined cautiously. Khunger warned that the hidden layers of agentic systems make them difficult to audit, suggesting a slow, phased adoption requiring extensive benchmarking against human performance and significant investment in testing infrastructure before these complex systems can be trusted with direct customer interactions. The overarching theme is that for financial AI to succeed near the customer, safety must be made visible and auditable across all dimensions.
🏢 Companies Mentioned
💬 Key Insights
"First, embedding safety into client-facing AI requires addressing technical vulnerabilities like prompt injection, model jailbreaking, and flawed training data right at the design stage."
"embedding safety into client-facing AI requires addressing technical vulnerabilities like prompt injection, model jailbreaking, and flawed training data right at the design stage."
"the specific functionality of working with systems that can jump between programs, perform searches, do deep research, know where to go within an organization to solve a complex customer inquiry, is a few years away from that for a lot of the reasons that you describe."
"you're probably going to see, especially from the vendor space, a lot of pop-up technologies that may claim they're agentic but really they're kind of glorified chatbots."
"while testing, decouple the processes and then try to see if the individual parts are giving the right output, not again look at the final results of that agentic AI process."
"the first challenge is, because there are so many hidden layers in agentic AI—agents working independently or interacting—it can get difficult to understand the model."