How this Yelp AI PM works backward from “golden conversations” to create high-quality prototypes using Claude Artifacts and Magic Patterns | Priya Badger
🎯 Summary
Podcast Episode Summary: How this Yelp AI PM works backward from “golden conversations” to create high-quality prototypes using Claude Artifacts and Magic Patterns
This 41-minute episode of “How I AI” features Priya Matthew Badger, a Product Manager at Yelp, detailing a novel, highly effective workflow for designing and prototyping AI-powered conversational products. The core innovation lies in starting the product definition process not with traditional requirements documents (PRDs), but by defining the ideal user experience through “golden conversations” generated and refined using Large Language Models (LLMs) like Claude.
1. Focus Area
The discussion centers on AI Product Management (AI PM), specifically the methodology for building effective conversational agents (like Yelp’s AI Assistant). Key technologies discussed include the use of Claude (Anthropic) for conversation simulation and prototyping, and Magic Patterns for visual interface exploration. The methodology bridges the gap between defining the conversational logic (the “behind the scenes”) and designing the user interface (the “interface”).
2. Key Technical Insights
- Golden Conversations as the First Artifact: The workflow begins by prompting an LLM (like Claude) to generate a complete, end-to-end sample conversation representing the desired user experience. This acts as the initial, content-rich wireframe, allowing PMs to work backward from the desired outcome.
- Claude Artifacts for Integrated Prototyping: Claude offers a unique feature allowing users to generate functional prototypes (Artifacts) directly from the refined conversations. These artifacts natively call the Anthropic API, enabling immediate, realistic simulation of the AI response flow, including latency and UI presentation, without complex API key setup.
- LLM Reasoning for Prompt Engineering: Analyzing the LLM’s “thought process” or reasoning output during initial tests is crucial. It helps PMs understand how the model is interpreting their inputs, aids in troubleshooting, and reveals the model’s emergent personality, which informs subsequent prompt refinement.
3. Business/Investment Angle
- Efficiency in AI Product Definition: By using LLMs to generate and iterate on “golden conversations,” Yelp significantly accelerates the early ideation and requirements gathering phase, moving past static documentation toward executable examples.
- Prototyping as Collaboration Tool: The ability to quickly generate shareable, interactive prototypes (via Claude Artifacts) enhances collaboration between PMs, designers, and engineers, allowing teams to feel the user experience (e.g., response length, waiting time) before significant engineering investment.
- Visual Exploration with AI-Assisted Design: Tools like Magic Patterns allow PMs to rapidly iterate on the visual interface (UI/UX) using natural language prompts, democratizing early-stage design exploration and speeding up the transition from conversational logic to visual mockups.
4. Notable Companies/People
- Priya Matthew Badger (Yelp PM): The central expert demonstrating the workflow.
- Claude (Anthropic): The primary LLM used for generating and simulating the “golden conversations” and creating functional prototypes via Artifacts.
- Magic Patterns: A prototyping tool used to visually explore UI/UX options based on the established conversational flow, leveraging AI to insert design elements via natural language.
- Colin Matthews: Mentioned as a previous guest who demonstrated recreating existing products in Magic Patterns.
5. Future Implications
The conversation suggests a future where the initial product requirements document for AI features is replaced by executable, high-fidelity examples (golden conversations). This shift emphasizes iterative, content-driven design, where the PM’s primary artifact is a simulation of the final user experience. Furthermore, the integration of LLM calls directly into prototyping tools signals a convergence of design, development, and AI logic generation.
6. Target Audience
This episode is highly valuable for AI Product Managers, Conversational Designers, UX Researchers focused on LLM interfaces, and Engineering Leads involved in building generative AI features. It offers a concrete, actionable playbook for moving from concept to testable prototype in the AI space.
Comprehensive Narrative Arc: Priya Badger introduces the unique challenges of AI PM: managing both the visible interface and the invisible, variable logic driven by LLMs. She outlines her team’s playbook, which centers on defining the desired experience through “golden conversations.” She demonstrates using Claude to role-play and generate these conversations, incorporating specific constraints (e.g., analyzing an uploaded photo of a cracked porch). This process is framed as an evolution of general product management best practices—prototyping with the artifact closest to the end-user experience.
After refining the conversational text through iterative feedback (e.g., making the AI more opinionated or budget-aware), Badger showcases the critical next step: using Claude Artifacts to instantly generate a functional, interactive chat application powered by the underlying LLM. This allows the team to test the feel of the conversation in a mobile-like context. Finally, she transitions to the visual layer, using Magic Patterns to take the established conversational flow and rapidly iterate on the front-end UI (e.g., exploring prompt suggestions like “Start with a photo” versus a simple camera icon). The entire workflow emphasizes using AI tools not just to build the product, but to define, refine, and prototype the product requirements themselves.
🏢 Companies Mentioned
💬 Key Insights
"What I like about AI is it's not just multi-modal and that you can put any sort of file type or data type in. It also allows you to approach problems from the front door, the back door, the side door, the window. You can come at your product problems in a much less linear way."
"You're starting from a kind of example consumer experience first. You're backing into kind of a rough prototype of what could support that experience. You're using an AI prototyping tool in this instance, Magic Patterns, to then put that experience in your brand and design guidelines. And then you're using that as a jumping-off point to fork and inspire a couple different versions of what that ultimate user experience could look like."
"You have demoed for us a completely new way of thinking about product management, prototyping, and product requirements in a way that is very different than I think what classic product management has looked at."
"And that manual iteration where it wasn't really moving the product forward, it was kind of getting our own minds around what the problem space and the solution space could be so that we could move the product forward, just took a lot of time. And so I think it's really interesting to compress the time for ideation so that you can get the ultimate product a little bit faster."
"In 2024, bot traffic officially surpassed human activity online. And with AI agents projected to drive nearly 90% of all traffic by the end of the decade, it's clear that most of the internet won't be human for much longer."
"I have to repeat it again for folks, you know, kind of starting inside out with a conversational agent prototyping example conversations first, getting them refined, getting a good set of example conversations that you can then put into a prototype generating tool in this instance, Claude, to then back into the chat experience, including the system prompt that would best serve those conversations."