This is the Ebbinghaus Forgetting Curve in action. Your brain isn’t a tape recorder; it’s a storyteller that fills in gaps with assumptions. If you’ve ever spent a week building a feature only to realize the VP never actually asked for it, you’ve experienced the “Shared False Memory” trap.

The Group Hallucination: Why Your Roadmap is at Risk

It’s one thing to forget a meeting; it’s another to collectively “invent” one. In groups, this phenomenon turns dangerous. Research shows that teams often walk out of a room with shared false memories, convinced the CEO issued a directive that she never actually said.

You implement a non-existent strategy, and six months later, no one even questions it because everyone “remembers” it clearly. I’ve seen a client team waste three entire weeks building a feature their VP supposedly “asked for”. The transcript later proved she never mentioned it, yet four senior people would have sworn on their careers that she did.

The hard truth?

Your memory is just a piece of fan fiction you wrote about the meeting after it ended.

The Failed Fixes of the Past

We’ve all tried the standard “productivity” hacks, and we know why they fail:

• Manual Note-taking: You split your focus. While you’re typing the last sentence, you’re missing the next one. You capture words, but you aren’t actually present to lead the conversation. And let’s be honest: you’ll never look at those notes again.

• The “Audio Graveyard”: You hit record, creating 47 minutes of audio that will sit in a folder until the end of time. We both know you aren’t going to replay it.

• The Bot Invasion: You invite an AI notetaker, and suddenly the vibe shifts. People talk differently when a bot is watching. You end up with a generic “nothing-summary” that tells you the team “discussed priorities”—which helps exactly no one.

You eventually give up and Slack a colleague: “Hey, what did we actually decide?”.

I’ll say it plainly: I hate AI notetakers.

You know the ones. You join a Google Meet or Zoom, and instead of a focused group of humans, you see a dozen “Assistants” and “Recorders” staring back at you. It’s a nightmare straight out of a Black Mirror episode.

When a bot joins, the room changes. People stop speaking freely. They filter their thoughts because they know a “gray circle” is transcribing their every word for a generic summary you probably won’t even read. We’ve reached a point where we should almost shame the use of these intrusive bots—they destroy the very “realness” of the conversation they’re trying to capture.

For a long time, I thought note-taking simply wasn’t a solved problem. I went back to scribbling poor, disjointed notes just to avoid the “Bot Invasion.”

Then, I found Granola

The “Invisible” Advantage: Why Granola is Different

Unlike typical AI notetakers, Granola is a local app. It doesn’t join your call as a participant. It simply listens to your device audio, transcribing in the background while you stay 100% present in the conversation.

• The "Ghost" Mode: No bot joins the call. Granola runs locally, capturing audio directly from your device. Because there is no "AI Notetaker" circle visible to others, the conversation remains natural and authentic. You get the real story, not the filtered version.

• Privacy First: Your data remains yours. Granola doesn’t store your audio—only the transcripts, which you control completely. This allows you to stop typing entirely and stay fully engaged in the meeting.

• Fully Focused: You don’t have to type a single word during the call. You can finally look your customers in the eye (digitally) and catch the nuance you’d usually miss while frantically taking notes.

• The Power of "Recipes": Beyond being invisible, Granola allows you to build custom prompts—what they call "Recipes"—to automate your most tedious tasks.

I know you’re already thinking about it: you want to copy my personal prompts. This is the #1 reason I’ve abandoned every other AI tool.

No More Copy-Paste Fatigue

The genius of Granola is its built-in AI chat. You don’t need to export text or juggle tabs with Claude or ChatGPT. You simply type “/” directly within your transcript to run a Recipe and get your output instantly.

<video>

Prompt 1: TLDR so-short-you-have-to-read it.

Default AI summaries are three paragraphs of nothing.

I need two bullets I’ll actually look at.

Link for direct access → https://recipes.granola.ai/r/7bc211bc-6278-4a78-a56b-d45630b72056 But if you want to recreate it, here’s the exact prompt I use:

You are summarizing a meeting for someone who is extremely busy and will only read something if it’s shockingly short. Your job is to distill the meeting provided into a single TLDR that is so brief, skipping it would feel ridiculous.

RULES:

• Maximum 2 bullet points total

• Each bullet must be under 12 words

• No fluff, greetings, or preamble

• No “In summary...” or “Key takeaways include...” — just the points

• Only include what actually matters: decisions made, actions required, or critical information

• If nothing important happened, say “Nothing worth your time.”

FORMAT:

• [Most important thing, the 80%]

• [Second most important thing, the 20%]

That’s it. Nothing else.

What you get back:

• Aman owns the pricing proposal by Friday.

• We’re not launching in Q1 anymore.

That’s it. The meeting in ten seconds.

Quick How-To for Granola Recipe.

How to create your own Granola Recipe:

1. Go to your Granola account.

2. Go to the bottom left “Recipe”.

3. Click on “Create a Recipe on the top right”.

And then it’s quite simple.

1. Type your name at the top (you will access it with / ).

2. The prompt part is the most important one. Copy mine or do yours.

3. Select the model. My favorite is claude-opus-4-5.

4. Make it for single meetings or multi-meetings.

PS: If you struggle to make your own Recipe, leave a comment.

I’ll help you prompt it.

Leave a comment

Prompt 2: Action items, nothing more.

Default action items are vague. “Follow up on discussion.”

Follow up on what? With who? By when?

I made a Recipe for that. My action items, nothing more.

The link for direct access → https://recipes.granola.ai/r/43347a99-e934-4972-8a88-cef1b9c7b27d

Or here’s the exact prompt of my Recipe:

You are reviewing a meeting to extract action items assigned to a specific person. Your job: Find and list of the action items assigned to the people the user will mention. It could be multiple people.

Rules:

- Output action items as bullet points only

- No introduction, greeting, summary, or closing remarks

- No headers or section titles

- If an action item has a deadline mentioned, include it in parentheses at the end

Example of an output format:

[name]

- Send the updated contract to legal (by Thursday)

- Book the venue for the offsite

- Review David’s proposal and share feedback

- Circle back with the client about pricing

Now I know exactly what I owe. Just what was actually said.

The “I never forget meeting” workflow.

1. Meeting starts.

2. Granola takes notes on the background.

3. Meeting ends.

4. Open Granola.

5. Type “/” → run the TLDR prompt → 2 bullets essence.

6. Type “/” → run the action items prompt → copy to Slack or Notion.

7. Done.

Two minutes. Every meeting.

No more “wait, what did we decide?”

You finally remember more than 30% after a day.

And you can still hate on AI notetakers bots. Shame them, really.

If you agree, say it louder in the comments.

We can all come up with great ideas for LLMs to enhance our products, but the devil is in the details:

• How do we get our AI to adopt a very specific writing style?

• How do we get our AI to use our latest product documentation?

• What’s the fastest way to test an AI concept and get it into users’ hands for feedback?

These aren’t just technical questions. They’re strategic product decisions.

Everyone can access powerful foundation models like GPT-4.5 or Claude Opus 4. So, what’s going to make your AI feature stand out?

Hint: It’s not just about having ‘AI’.

It’s about making that AI uniquely yours and genuinely useful.

The 3 Approaches to Optimizing AI

Base LLMs are like brilliant interns: incredibly capable, but they don’t know your company’s specific jargon, your proprietary data, or the nuanced style your customers expect.

Leaving them “off-the-shelf” often leads to:

• Hallucinations & Generic Answers

• Lack of Domain Expertise

• Inconsistent Brand Voice

You have three major ways to address these problems:

1. RAG (Retrieval Augmented Generation)

   Have a model do a search to enhance its results with information not in its training data set, then incorporate those findings into its answer

2. Fine-tuning

   Specializing a model based on information

3. Prompt engineering

   Better specifying what you’re looking for from the model

The question is: when should you use each, and why?

Today, we’re breaking down the definitive framework for choosing between prompt engineering, RAG, and fine-tuning.

Today’s Post

We’re going to give you the context, decision frameworks, and a practical step-by-step walkthrough to help you build the intuition. Once you build each on your own, you’ll have that AI engineer level knowledge to speak with them confidently:

1. Mistakes

2. Pros and Cons

3. Decision Framework

Think of this post more like a course-lesson where you have to follow along than an article.

TL;DR: Start with prompt engineering (hours/days), escalate to RAG when you need real-time data ($70-1000/month), and only use fine-tuning when you need deep specialization (months + 6x inference costs).

1. The 3 Major Mistakes We See

Let’s start by understanding what not to do, then build up to the right approach.

The Strategic Imperative

The truth? There’s a strategic choice between Prompt Engineering, RAG, and Fine-Tuning.

Choosing RIGHT? That’s your path to AI-driven product growth.

Too often, teams jump to:

1. Over-engineering: “Let’s fine-tune a massive model!”

   (Costly, time-consuming, maybe overkill)

2. Under-powering: “We’ll just tweak the prompt a bit more.”

   (Hits a ceiling, can’t handle complex needs or new data)

3. The “Black Box” Hope: “We’ll plug in an API and hope for the best.”

   (Lacks control, struggles with proprietary knowledge)

The truth is, there’s a strategic choice to be made between Prompt Engineering, Retrieval Augmented Generation (RAG), and Fine-Tuning.

Choosing wrong means wasted cycles, blown budgets, and features that underwhelm.

Choosing right? That’s your path to AI-driven product growth.

This article is your map.

2. Pros and Cons: RAG vs. Fine-tuning vs. Prompt Engineering

Let’s start by understanding each of the three techniques at a deep conceptual level and discuss their pros and cons.

Prompt Engineering

As upcoming podcast guest Hamel Husain says:

Prompt engineering is just prompting these days.

We all have to prompt, but when it comes to building the right prompt for your product feature, prompt engineering is critical.

It goes beyond simple clarification. It’s about transforming the model’s output with additional training or data retrieval. It’s about better activating a model’s existing capabilities.

Pros:

• You don’t need to change backend infrastructure

• You get to see immediate responses and results to what you do - no new training data or data processing required

Cons:

• Trial and error - it’s as much an art as a science

• You’re limited to the model’s existing knowledge, unable to add new or proprietary data

Fine-tuning

Fine-tuning takes an existing foundation model and gives it specialized ‘graduate-level’ training on a focused dataset relevant to your specific needs.

You’re subtly adjusting the model’s internal ‘weights’ (its understanding of relationships in data) to make it an expert in a particular domain, style, or task.

This typically involves providing hundreds or thousands of high-quality input-output examples.

Pros:

• Great when you need deep domain expertise or consistent tone/style

• Faster at inference time than RAG because it doesn’t need to search through external data and don’t need to maintain a separate vector database

Cons:

• Issues with the training complexity - need 1000s of examples

• There are significant computational and maintenance costs involved

• You risk “catastrophic forgetting,” where the model loses some general capabilities as it becomes more specialized

RAG

Retrieval Augmented Generation is like giving your LLM real-time access to a specific, curated library of information – your product docs, a knowledge base, recent news, etc.

When a user asks a question, the RAG system first retrieves relevant snippets from this external library and then feeds that context to the LLM along with the original query.

The LLM then uses this fresh, targeted information to generate its answer.

Pros:

• Good for up-to-date information

• Good for adding domain-specific information

Cons:

• Performance impact - retrieval adds latency to each prompt (typically 100-500ms)

• Processing costs - eg for the vector database

Ready to add more context to this? We highly recommend staying around for section 4 - where we walk you step-by-step through all 3 so you can have a great addition to your PM portfolio and learn the intuition behind this.

3. Decision Framework

Now comes the million-dollar question: how do you actually decide which one to use, and when?

This is our battle-tested framework after learning from case studies at Shopify, Google, Apollo, and OpenAI:

The Million-Dollar Mistake Most Teams Make

Here’s the uncomfortable truth, the one that burns through budgets and timelines faster than a poorly optimized LLM: Teams reflexively jump to the most complex solution.

Fine-tuning sounds sophisticated, like you’re truly ‘building AI.’

RAG systems feel robust, like you’re tackling data head-on.

So, engineers get excited, data scientists propose intricate architectures, and suddenly, you’re six months deep into a project that could have delivered value in six weeks.

Worst of all? They try to do everything at once.

At Shopify, we’ve seen this play out firsthand with ‘Auto Write,’ their AI content generation feature.

The initial internal buzz was all about fine-tuning GPT-3. Data science was already sketching out a sophisticated RAG system. Everyone had a strong opinion, usually leaning towards the most technically challenging path.

What actually worked? What shipped and delivered value?

They started with disciplined prompt engineering. Simple, direct, and focused.

The result?

They shipped a high-accuracy feature in 10 weeks.

The fancy, more complex approaches like RAG and fine-tuning came later – much later – only after they had validated the core user problem and proven the initial value with the simplest effective method.

The lesson is: don’t let the allure of complexity derail your path to impact.

Our Practical Decision Framework: Your Step-by-Step Guide

How do you avoid the ‘Million-Dollar Mistake’ and make the right call? Here’s the battle-tested decision framework Miqdad and I use:

Let’s walk through the logic.

Step 1: Nail Your Use Case

Before you write a single line of code or craft a prompt, ask yourself: Is the user problem you’re aiming to solve with AI crystal clear? Do you know precisely who this is for and what a “win” looks like from their perspective?

Vague goals breed vague AI – features that meander, require endless tweaks, and ultimately, satisfy no one.

“AI for sales” isn’t a strategy; “AI to draft personalized follow-up emails for SMB sales reps based on CRM interaction history” is getting warmer.

If your use case is fuzzy, stop. Seriously. Hammer out the problem definition, user stories, and key success metrics.

This isn’t just a nice-to-have; it’s the non-negotiable foundation. Only once your target is sharp should you proceed.

Step 2: The Prompt Engineering Gauntlet

With a clear use case, your first port of call is always prompt engineering. Can you get to roughly 80% of your desired outcome with well-crafted prompts alone?

This is where you test your MVP or prototype ideas. It’s your fastest, cheapest way to see what the base model is capable of.

Don’t underestimate what good prompting can achieve.

But also, don’t fall into the trap of spending months trying to coax magic out of prompts for a task that fundamentally needs more.

Give yourself a tight timebox – say, one to two weeks max of dedicated effort – to explore this. If you hit that 80% mark, fantastic! Ship it. Get it in front of users, gather feedback, and iterate.

Step 3: The Data Question – Does Your AI Need a Live Feed?

If prompts alone aren’t cutting it, the next question is about data.

Does your AI feature absolutely depend on information the base model couldn’t possibly know? Think: data that changes by the minute, your company’s latest internal product documentation, or user-specific context.

Trying to “teach” an LLM constantly shifting facts through fine-tuning is like trying to fill a leaky bucket – slow and inefficient.

And while prompts can handle small bits of context, they can’t effectively inject vast, ever-changing datasets.

If your AI needs to be current and deeply aware of your specific, evolving world, then RAG (Retrieval Augmented Generation) is your answer.

It gives your LLM a live connection to the knowledge it needs, precisely when it needs it. If external data isn’t the core issue, your challenge likely lies in shaping the model’s inherent behavior or style.

Step 4: The Fine-Tuning Fuel Check – Got Quality Data?

So, prompts weren’t enough, and external data access (RAG) isn’t the primary bottleneck.

You’re now likely looking to instill a very specific style, tone, or a complex, nuanced understanding that goes beyond simple information retrieval.

This is often the case for achieving high volume and consistency in outputs, or for tackling enterprise-level tasks demanding sophisticated pattern recognition.

Welcome to the doorstep of fine-tuning.

But before you step through, check your fuel tank. Do you have at least a thousand (and often many more) high-quality training examples?

These aren’t just any examples; they need to be carefully curated, accurate input-output pairs that perfectly demonstrate the desired behavior you want the model to learn.

Fine-tuning with a handful of examples, or with messy, low-quality data, is a recipe for disaster.

Garbage in, garbage out is brutally amplified here. If you don’t have this data, your immediate focus should be on collecting and curating it, even while you continue to iterate with prompts or RAG.

Step 5: The ROI Litmus Test for Fine-Tuning

You’ve got the data. The need seems clear. But now, the crucial business question: Is the return on investment for fine-tuning undeniably compelling?

Fine-tuning “because we can” or because it sounds impressive on a slide deck is how innovation budgets die a quiet death. The bar should be high.

If the value proposition isn’t screamingly obvious, it’s probably not the right time. Stick with your refined prompt and RAG setup. Re-evaluate later when the case is stronger.

But if the data, the need, and the ROI all align? Then you’ve earned the right to strategically deploy this powerful, resource-intensive technique.

The Core Philosophy: Start Simple, Add Complexity Deliberately

The takeaway here isn’t subtle: Always start with the simplest approach that could work.

Prompt engineering is your first, best friend. Only escalate to RAG or fine-tuning when the problem genuinely demands it, and you can clearly articulate the business case for the added complexity and cost.

Stay tuned for Part 3…

We have mastered instant home deliveries. We have solved door-to-door travel. We have built entire industries around helping people earn and save money.

But what happens after we save that time?

Often, a second wave of companies—social media, endless streaming, doom-scrolling apps—sweeps in to capture that “saved” time.

We saved minutes on groceries only to lose hours to distraction.

Even education companies, which promise better careers and higher salaries, often result in corporate friends who are burnt out and unhappy.

The actual problem isn’t efficiency. It’s happiness.

And frankly, most technology today doesn’t know how to solve that.

In that post, I suggested three starting points to fix this:

1. Mental Therapy (Access to care)

2. Humanizing Business (Empathy in work)

3. Tech Empowerment (Tools that heal rather than distract)

Today, I want to talk about that third point: Tech Empowerment.

Subscribe now

Building the Solution

As we step into the New Year, I didn’t want to just make resolutions. I wanted to build a tool that helps keep them.

I realized that while I cannot replace a human therapist, I can use the latest advancements in AI agents to make support more accessible, immediate, and structured.

This week, I built a Multi-Agent Mental Wellbeing System.

Unlike a standard chatbot that gives you generic advice, this system mimics a supportive care team. It is designed to help you deal with the chaos of life—whether it’s job stress, relationship anxiety, or just that overwhelming feeling that comes with a new year.

How It Works (and How It Helps You)

The tool isn’t just one AI; it is a “swarm” of three specialized agents working together to support you:

1. The Listener (Assessment Agent): We often just need to be heard. This agent doesn’t try to “fix” you immediately. It listens. It analyzes your emotional state, sleep patterns, and stress levels to understand exactly where you are standing right now.

2. The Helper (Action Agent): Once you’ve been heard, you need clarity. This agent cuts through the noise. It provides immediate, concrete steps—whether that’s a breathing exercise for right now or a resource to call tomorrow. It turns anxiety into action.

3. The Partner (Follow-up Agent): This is the piece most tech misses. We don’t just need a quick fix; we need a lifestyle change. This agent builds a long-term strategy for you, focusing on habits and sustainable routines.

A Tool for the New Year

The New Year is often a source of pressure. We set massive goals and feel like failures by February.

I want this agent to be a tool for resilience, not just productivity. It is an example of technology that doesn’t demand your attention but supports your intention.

If you are feeling the weight of the world, or just the weight of a hectic job, give it a try. Let’s use technology to reclaim not just our time, but our peace of mind.

Here’s to a happier, healthier, and more empowered 2026.

Adityeah

Subscribe now

P.S. For the Builders

For those interested in the architecture, this system uses AG2 (formerly AutoGen) to orchestrate the agents. By separating “Empathy” (Assessment) from “Logic” (Action), we drastically reduce hallucinations and improve the quality of care.

You can check out the open-source code here:

GitHib Link

In one world, the tech has never been better. Chips are faster, models are stronger, and building AI features is easier than ever.

In the other world, the features we ship often fall short. Many AI tools still feel like junior interns—helpful at times, but needing guidance and making mistakes you don’t expect.

And most of the time, the issue isn’t the model itself. It’s the context we give it.

That’s why context engineering matters. It may sound technical, but it’s quickly becoming one of the most important skills for anyone building AI products.

Today’s Post

1. Why Does Context Engineering Matter?

2. The PM’s Role in Context Engineering

3. The 6 Layers of Context to Include

4. Common Mistakes

5. How to Engineer Context Step-by-Step

6. How to Spec out Features Appropriately

7. Checklists, templates + prompts you can steal

1. What is Context Engineering, and Why Does it Matter?

What Is Context Engineering?

Context engineering is the practice of giving an AI model the exact information it needs at the exact moment it needs it. Think of prompt engineering as writing instructions, while context engineering builds the world those instructions live in.

Humans naturally infer context—we remember who we’re talking to, what just happened, what matters, and what rules apply. LLMs don’t. The models don’t magically know:

• who the user is

• what happened a few seconds ago

• which document matters

• what the system already knows

• what rules or policies apply

• what data they’re allowed to access

• what was said in previous sessions

• whether the user is new or experienced

• what constraints must be followed

• which entities exist in the workspace

• how different pieces fit together
  

None of this comes for free. We have to supply it.

That is the job of context engineering—and it’s the difference between an AI that feels clueless and one that feels intelligent.

Why Context Engineering Matters?

Most people obsess over models and prompts.

“Should we use GPT-5.1 or Gemini 3?”
“What’s the perfect prompt?”

Those questions matter—but only after you’ve done the hard work of context.

Because if your system:

• doesn’t know which file the user is working on

• doesn’t see the user’s preferences

• isn’t aware of entities and relationships in the workspace

• can’t recognize the user’s role

• retrieves random, irrelevant documents

• or misses key logs and events

…then it doesn’t really matter which model you plug in. The output will still feel off.

Context is what turns “a smart model” into “a useful product.”

Example 1: AI Email Assistants

When I was working on an AI email writer, I learned this the hard way.

What the model saw changed everything:

• Only the last message → the reply was generic

• Full email thread → the reply became coherent

• Thread + CRM notes → the reply became personalized

• Thread + CRM + tone-of-voice guidelines → the reply felt on-brand

• Thread + CRM + tone + relationship context (e.g., “key customer,” “renewal risk”) → the reply became shippable

Same model. Very similar prompts. Completely different quality.

That’s context engineering at work.

Example 2: AI Coding Assistants (Cursor)

Take coding assistants.

Ask yourself: why hasn’t Cursor been steamrolled by every new “better” model release, and instead grown into a billion-dollar+ business?

The key is not just which model they use. It’s how they prepare context.

Roughly speaking, tools like Cursor:

1. Index your codebase:

   • Split the code into meaningful chunks (often using the AST).

   • Compute embeddings for every chunk.

2. Understand your question:

   • Turn your query into embeddings.

   • Search a vector database for the most relevant files, functions, and lines.

3. Build a rich context window:

   • Pull in only the relevant snippets.

   • Feed those into the model along with your request.

Because the context is so strong, Cursor can let you switch between OpenAI, Anthropic, Gemini, xAI, etc. The model is almost a plug-in.

The moat is the context layer: the indexing, retrieval, and wiring of code + queries into something the model can reason over.

That’s also why you see huge companies trying to buy or replicate these players instead of just “turning on a better model.” Model capability alone is not enough. Context is what makes the product defensible.

Now that we’ve talked about why context engineering matters and how it shows up in real products, the rest of this guide will focus on the “how”:

• frameworks and canvases you can use as an AI PM

• real-world checklists

• templates and prompts you can copy straight into your own products

The goal: help you treat context engineering as a first-class product skill, not an afterthought.

2. The PM’s Role in Context Engineering

Many teams assume context engineering is something only engineers handle.

It isn’t.

Context engineering sits where product thinking, user insight, and system design meet. Engineers can build the plumbing, but they can’t decide which signals matter, how they shape the experience, or what “good” looks like for the user.

That judgment lives with the PM.

What PMs Own in Context Engineering?

As a PM, there are three layers you’re directly responsible for—layers engineers cannot decide on their own.

1. Defining what “intelligence” means for the feature

Before anything is built, you must spell out what success looks like. That includes:

• What the AI should know about the user

• What domain knowledge is essential vs. optional

• Which user actions should update the context

• How much personalization creates value without crossing the line

These aren’t technical calls—they’re product calls.

2. Translating user value into context requirements

Users express fuzzy needs. PMs convert those into concrete context specifications.

• “Users want better suggestions” →
  system needs access to past dismissals, workspace state, and team preferences

• “Make it feel personalized” →
  capture writing style, common corrections, and role-specific patterns

Without this translation, engineering builds general-purpose systems that function but don’t feel smart.

3. Designing how the system behaves when context is missing

Context will fail: documents won’t load, signals will be stale, permissions will restrict data.

You decide the fallback behavior:

• Should the feature block?

• Should it return a partial answer?

• Should it ask clarifying questions?

• Should it switch to a simpler mode?

This is core product judgment. It defines reliability, trust, and user perception.

What Engineers Own?

Engineers handle the execution:

• retrieval architecture

• vector databases

• embeddings and chunking

• indexing pipelines

• integrations

• latency, reliability, and scaling

But all of this depends on the PM defining what to retrieve and why it matters.

The Division of Labor

A simple way to think about it:

• PMs define the context pyramid — what information belongs at each layer

• Engineers build the infrastructure — how to store, fetch, and update it

• PMs design the orchestration logic — what the model should see and when

• Engineers implement the orchestration engine — the system that delivers it in real time

When PMs skip their part, teams end up with technically impressive systems that feel oddly unintelligent—because nobody defined what intelligence actually required.

3. The 6 Layers of Context to Include

If you look closely at every world-class AI product, you’ll find their intelligence doesn’t come from clever prompting or bigger models. It comes from a carefully engineered hierarchy of context.

LAYER 1 — INTENT CONTEXT

Starting at the foundation…

Deciphering the user’s actual goal, not just their literal words.

Nearly every AI disaster—hallucinations, logic errors, bad suggestions, irrelevant answers—stems from a single failure point: the system misread the user’s objective.

Humans are messy communicators.

They say one thing but want another. They highlight a paragraph without explaining the error. They click, pause, rewrite, or use shorthand that relies on assumptions the model cannot see.

This is the role of Intent Context: to give the AI interpretive intuition—the ability to reconstruct the user’s true aim, even when the prompt is vague, incomplete, or emotional.

The “Triple-I” Decryption Model

To turn intent from a concept into code, every input must pass through a three-stage filter:

1. Interpret: Convert the raw text into a clear, structured objective.

2. Infer: Analyze recent actions (clicks, hovers, deletions, retries) to find the subtext.

3. Identify Gaps: Spot missing data points (files, code snippets, metrics) that must be fetched before answering.

This converts messy input into machine-ready intent, clearing the fog so the next five layers can work.

LAYER 2 — USER CONTEXT

A live profile of the human: their habits, quirks, role, and mental model.

Even a technically perfect answer fails if it feels robotic.

A principal architect wants the raw diff; a junior developer needs the tutorial. A CEO wants the bottom line; a researcher wants the citations.

User Context ensures the AI acts not as a generic chatbot, but as a personalized extension of the operator—matching their rhythm, tone, expertise, and history.

The 5-P Personalization Matrix

To make this practical, track these five variables for every user:

1. Preferences: Voice (terse vs. warm), verbosity, style.

2. Patterns: How they usually edit, format, or correct outputs.

3. Proficiency: Expert vs. Novice (dynamic complexity adjustment).

4. Pacing: The speed at which they consume information.

5. Purpose: Their specific job-to-be-done and workflow goals.

This allows the AI to speak in the user’s language, rather than the LLM’s default setting.

LAYER 3 — DOMAIN CONTEXT

Without grounding, an AI isn’t a product; it is a hallucination engine with a nice UI.

Domain Context transforms your system from a creative toy into a subject-matter expert. It enables the AI to cite real objects, respect dependencies, use correct terminology, and navigate your company’s reality.

This is the “Source of Truth”: your entities, metadata, dependency trees, documentation, codebases, business logic, and institutional history. But you cannot just dump raw text. It must be structured.

Every domain requires five structural anchors:

1. Entities: The nouns of your world: tickets, users, dashboards, repos.

2. Attributes: The descriptors: dates, owners, tags, status.

3. Relationships: The web: “blocks,” “enables,” “owns,” “is part of.”

4. Definitions: The logic: canonical formulas and business rules.

5. Lineage: The history: who changed what, and when.

LAYER 4 — RULE CONTEXT

The laws, limits, and guardrails that dictate what the AI allows.

Intelligence without constraints is dangerous. Rule Context acts as the judiciary of your system: deciding what is permitted, what is required, and what is strictly banned.

This covers everything from output schemas and permissions to safety compliance and formatting mandates. These cannot be polite suggestions in a prompt—they must be hard boundaries.

The Dual-Barrier Framework

Enforce rule context using two types of walls:

1. The Soft Wall: Advisory guidelines (brand voice, style, tone).

2. The Hard Wall: Mandatory requirements (JSON schemas, security, permissions).

Soft walls guide the vibe. Hard walls guarantee the function.

Together, they turn the AI from a probabilistic guesser into a deterministic operator.

LAYER 5 — ENVIRONMENT CONTEXT

The immediate reality surrounding the user right now.

Most work depends on the “now,” not on static knowledge.

A coding bot needs to see the open tab. A writing assistant needs the selected sentence. A support bot needs the error log from one minute ago. A planning bot needs the deadline expiring today.

Environment Context gives the AI situational awareness—knowing the active file, the cursor position, the timestamps, and the current system state.

The N.O.W. Awareness Model

Capture the real-time signal via three channels:

1. Nearby Activity: The user’s immediate focus (selection, cursor, scroll).

2. Operational State: System health, logs, errors, device info.

3. Window of Time: Urgency, deadlines, and recency.

This stops the AI from acting generically and helps it act contextually—like a coworker looking over your shoulder.

LAYER 6 — EXPOSITION CONTEXT

The final, polished, noise-free signal the model actually receives.

This is the top of the stack—the point where the previous five layers merge into a clean, ranked, coherent payload.

Exposition Context is where potential becomes performance.

It is the difference between handing the model a library and handing it the exact page it needs.

The Context Distillation Pipeline

Every prompt payload must survive a five-step purification process:

1. Collect: Aggregate signals from Intent, User, Domain, Rule, and Environment.

2. Compress: Cut the fluff, remove redundancy, resolve conflicts.

3. Construct: Organize the data into clear, labeled sections.

4. Constrain: Apply safety checks and formatting rules.

5. Check: Verify the payload is consistent and ready for reasoning.

Unifying the Stack

When these six layers align, the AI wakes up. Most products skip steps or build them poorly. The winners treat every layer as critical infrastructure, not optional features.

4. The Divide: Engineering Context vs. Dumping Data

If you analyze high-performing AI agents, you will find they share a distinct architectural blueprint. Conversely, systems that hallucinate, drift, or confuse users almost always suffer from the same set of structural flaws.

The difference isn’t the model; it’s the discipline of the context pipeline.

What Great AI Products Do Right

World-class AI products treat context as a strict engineering discipline, adhering to these principles:

1. Structure Over Chaos: They never feed the model raw blobs of text. They invest heavily in schemas, metadata fields, and entity graphs because they know that unstructured inputs inevitably yield unstructured behavior.

2. Surgical Curation: They do not “give the model everything.” They aggressively pre-filter, selecting only the specific signals required for the current reasoning step. They value signal-to-noise ratio over total volume.

3. Explicit Segmentation: The model receives context in clear, labeled packets (e.g., <UserHistory>, <SystemConstraints>, <ActiveFile>). They use the six layers to feed information in a predictable hierarchy, reducing ambiguity.

4. Deterministic Guardrails: They do not rely on the LLM to police itself. Hard constraints are enforced by external validators and code logic, not by polite requests buried in the prompt.

5. Dynamic Knowledge Graphs: They treat domain knowledge as a living network. Every new document or interaction updates the relationships in the graph, ensuring the context evolves alongside the business.

6. Multi-Step Reasoning: They rarely rely on a single “God Prompt.” Instead, they use chains: planning steps, summarization calls, and iterative refinement loops to build the answer piece by piece.

The Architecture of Failure

Broken systems are equally predictable. Across hundreds of failed implementations, these patterns appear repeatedly:

1. The “Kitchen Sink” Strategy: Stuffing the entire context window with every available document. This leads to token overload, model confusion, and “lost in the middle” retrieval errors.

2. Naive Semantic Search: Relying solely on vector search (RAG) without structural filtering. This retrieves conceptually similar but factually irrelevant chunks that derail the model’s logic.

3. The “Prompt-as-Code” Fallacy: Assuming that writing “Do not do X” in a prompt is a security measure. Prompts are probabilistic suggestions; without hard code boundaries, the model will eventually ignore them.

4. Source Amnesia (No Provenance): Feeding data to the model without tracking where it came from. When the AI cannot cite its sources or distinguish between an old memo and a new policy, reasoning becomes impossible.

5. The “Magic Box” Assumption: Expecting the LLM to infer the domain structure on its own. No matter how smart the model, it cannot guess your internal business logic unless you explicitly define the schema.

The Bottom Line

Great context engineering is about discipline. It requires rejecting the shortcut of “just dumping data” into the prompt and instead building the rigid infrastructure—schemas, layers, and filters—that true intelligence requires.

5. How to Engineer Context Step-by-Step The C.E.O. Framework

The previous section covered what context is. This section covers how to build the invisible machinery that manages it.

Truly intelligent AI doesn’t just “react” to a prompt. It runs a continuous background process that gathers raw signals, structures them into meaning, and carefully curates what the model sees.

This is the C.E.O. Framework: Capture, Enrich, and Orchestrate.

PHASE 1: CAPTURE

How the System Absorbs the World

“Capture” is the foundational act of noticing. It is the sensor array that collects the raw ingredients of intelligence from the user, the application state, and the environment.

In high-quality AI products, this happens passively. The system does not wait for the user to explain their situation; it silently observes the micro-signals that define intent. It notices:

• The document currently open.

• The specific text highlighted or the field just edited.

• The filters applied to a dashboard.

• The segment drilled down into during analysis.

• The last output the user accepted (or rejected).

• Looming deadlines on the timeline.

The Three Streams of Signal To build a complete picture, a robust capture layer aggregates three distinct families of inputs:

1. Explicit Signals: What the user intentionally provides (prompts, specific selections, uploaded files).

2. Implicit Signals: What the user subconsciously reveals (scroll depth, cursor hover time, recent click paths).

3. System Signals: The environmental reality (timestamps, object metadata, data freshness, error logs).

The Goal: To transform a stream of fragmented micro-data into a coherent answer to the question: What is actually happening right now?

PHASE 2: ENRICH

How Raw Signals Become Structured Meaning

Raw data is noisy; enriched data is actionable.

Enrichment is the process of converting ambiguous, unstructured signals into a consistent schema the model can reason about. It turns the messy complexity of human behavior into a clean “context package.”

This involves extracting entities, normalizing fields, resolving ambiguity, and filtering noise. It is the difference between the system seeing “User clicked the button” and seeing “User approved the Q3 Financial Report.”

The Engineering Challenge: Reconciling Reality Enrichment is difficult because reality is inconsistent. The system must handle:

• Ambiguity: Two users using the same phrase to mean different things.

• Vague References: A user asking about “the latest update” (which requires checking timestamps against version history).

• State Drift: A metric changing in the background while the user was working on something else.

The Transformation Pipeline A sophisticated enrichment layer performs four critical operations:

1. Disambiguation: Clarifying references like “this issue” or “that customer.”

2. Stitching: Connecting related objects (e.g., linking a PR to the Incident Ticket it solves).

3. Inference: Reconstructing implied relationships (e.g., “The user selected a high-risk item, implying they are in a risk-mitigation workflow”).

4. Augmentation: Injecting domain knowledge (e.g., “This metric relies on Stream X, which is currently down”).

PHASE 3: ORCHESTRATE

How the System Decides What the Model Sees

Orchestration is judgment.

Just because you captured and enriched the data doesn’t mean the LLM should see all of it. Feeding a model too much information confuses it; feeding it too little hallucinates it.

Orchestration is the deliberate decision-making layer that determines exactly which artifacts are included, in what order, and in what format.

The Four Balancing Forces The orchestrator must balance four competing constraints simultaneously:

1. Relevance: Does this signal materially affect the answer, or is it noise?

2. Brevity: Can we fit this within the token window without losing nuance?

3. Precision: Is the structure clear enough for the model to parse?

4. Timing: Is this the right information for this specific step of the chain?

The Architecture of Control This is where context engineering becomes an art form. A strong orchestration layer separates concerns into distinct logic blocks:

• The Filter: Selects the most relevant artifacts from the domain graph.

• The Structurer: Organizes data into labeled sections (summaries vs. raw text).

• The Ruler: Enforces constraints and business logic.

• The Personalizer: Injects user history and preferences.

• The Flow Controller: Decides if the model needs a planning step or a retrieval call before generating the final answer.

The Result: The model receives a highly curated, coherent dataset designed specifically for reasoning—not a random blob of text.

6. How to Spec out Features Appropriately: The 4D Context Canvas

You now have the Context Pyramid (the categories of context) and the C.E.O. Framework (the engine to process it).

But when you sit down to design a specific feature, you need a tactical plan. You need to answer: What specific context does this feature require? Where does it live? And what happens when it’s missing?

This is the role of the 4D Context Canvas.

Most AI features do not fail because the model wasn’t smart enough. They fail because the team never explicitly defined the context required to succeed. They relied on dangerous assumptions like “we’ll just fetch that data” or “the model will figure it out.”

To prevent this, every AI feature must be stress-tested against four dimensions:

1. Demand: What is the model actually being asked to do?

2. Data: What context is required to get it right?

3. Discovery: How will we obtain that context reliably during runtime?

4. Defense: How will we detect failures and prevent incorrect outputs?

D1 — DEMAND

Defining the Model’s Job Precisely

If you cannot articulate the job, the model cannot perform it.

The primary cause of generic, erratic AI behavior is vague product requirements. Developers often write prompts without a clear specification, forcing the system to guess the user’s goal.

To fix this, you must translate fuzzy intent into a rigid Model Job Spec. You must clarify exactly what is being produced, for whom, and under what constraints.

The Transformation

• The Vague Request: “Draft a status update.”

• The Precise Demand: “Summarize key changes in Project X since the last report, formatted for Stakeholder Y, adopting the user’s concise tone, and strictly adhering to the weekly reporting schema.”

The Spec Checklist A proper demand spec must explicitly define:

1. Inputs: What raw materials will the model receive?

2. Assumptions: What acts as the “default” truth if data is missing?

3. Required Outputs: The exact format, length, and structure.

4. Success Criteria: What separates a “correct” answer from a “plausible” one?

D2 — DATA

Mapping the Hidden Dependencies

Once the job is defined, you must identify the fuel.

Every AI feature has hidden dependencies. The Data step forces you to map the exact context required to execute the job. This is best done via a Context Requirements Table—a simple audit that removes ambiguity.

The Context Audit For every piece of context, you must define four attributes:

1. The Signal: What entity, document, metric, or object is needed?

2. The Source: Where does it live? (SQL DB, Vector Store, API, Browser State).

3. Availability: Is this data always there, sometimes there (dependent on user history), or rarely there?

4. Sensitivity: Is this PII? Internal-only? Public?

Example: The AI Sprint Planner To plan a sprint, the model cannot just “know” agile. It needs:

• Backlog Items (Source: Jira API | Availability: High)

• Team Capacity (Source: HR System | Availability: Variable)

• Historical Velocity (Source: Analytics DB | Availability: High)

• Cross-Team Dependencies (Source: Knowledge Graph | Availability: Low/Hard to fetch)

When you map this honestly, you immediately see if your feature is viable or if you are missing the data pipelines to support it.

D3 — DISCOVERY

The Runtime Retrieval Strategy

Knowing what data you need is easy. Getting it in real-time is hard.

Discovery defines the logistics: how will the system locate, retrieve, and assemble the context at the exact moment the user presses “Enter”?

You must engineer a retrieval strategy that balances precision with latency:

The Retrieval Toolkit

• Search-Based: Using Vector Search for semantic themes or Keyword Search for exact matches (e.g., retrieving specific error codes).

• Graph-Based: Navigating the Knowledge Graph to find related entities (e.g., “Find the owner of the project that this ticket belongs to”).

• Precomputed: Using cached views or daily jobs to have heavy data ready instantly (e.g., “Yesterday’s sales totals”).

The Trade-off: Speed vs. Depth Teams must make hard choices here. Which context must be real-time? Which can be 24 hours old? Which data is too expensive to fetch for a sub-second interaction?

D4 — DEFENSE

Guardrails, Fallbacks, and Failure Modes

A feature is not ready until you have designed its failure state.

Defense is the layer that turns a fragile demo into a resilient production system. Because in the real world, APIs will time out, data will be stale, and models will hallucinate. Defense is about catching these failures before the user sees them.

The Four Defensive Layers

1. Pre-Checks (The Gatekeeper): Before invoking the model, ask: “Do we actually have enough context to answer this?” If the required entity is missing or the data is too old, block the generation and ask the user for clarification.

2. Post-Checks (The Auditor): After the model generates text, validate it. Does it match the JSON schema? Did it violate a safety policy? Is it logically consistent with the source data?

3. Fallback Paths (The Safety Net): When the system breaks, degrade gracefully. Do not show a stack trace.

   • Instead of a hallucination: Offer a conservative summary.

   • Instead of silence: Ask a clarifying question.

   • Instead of a crash: Revert to a hard-coded default.

4. Feedback Loops (The Teacher): Capture the signal from failure. Use explicit ratings (thumbs down) and implicit behavior (user editing the AI’s output) to detect patterns and patch the holes in your context logic.

   ---

   7. The Toolkit: Checklists, Templates, and Prompts

   You now have the theory and the architecture. This section provides the copy-paste assets you need to deploy them.

   Below is the Context Quality Checklist to run before every API call, and the Orchestrator Meta-Prompt that serves as the backbone for high-reasoning systems.

   Part 1: The Context Quality Checklist

   Use this “Pre-Flight Check” every single time before sending a payload to an LLM.

   Hallucinations are rarely model failures; they are almost always context failures. If you feed the model noise, ambiguity, or contradictions, no amount of “prompt engineering” will save you.

   Use this five-point audit to ensure the “brains” of your system are fed clean fuel.

   1. THE RELEVANCE AUDIT (Signal-to-Noise)

   • Curated: Does every piece of text in the context directly help answer the specific intent?

   • Distilled: Have you stripped away “kind of related” documents that dilute the model’s focus?

   • Cleaned: Did you remove decorative JSON metadata (internal IDs, unrelated flags) that wastes tokens and confuses the model?

   2. THE FRESHNESS AUDIT (Temporal Validity)

   • Timestamps: Are the documents and logs from the current relevant window?

   • State Awareness: Does the context reflect the system as it is right now (not cached from an hour ago)?

   • Deprecation: Have you filtered out “legacy” or “archived” entities that might mislead the model?

   3. THE SUFFICIENCY AUDIT (The “Missing Link” Check)

   • Entity Completeness: If the user asks about “Project X,” did you include Project X’s owner, status, and recent comments?

   • Dependency Mapping: Did you include related objects (e.g., the blocker preventing the project from moving)?

   • Gap Analysis: If a critical piece of data is missing, does the system know it is missing? (Preventing the model from guessing).

   4. THE STRUCTURE AUDIT (Parsability)

   • Segmentation: Is the context broken into clear, labeled sections (e.g., <History>, <Rules>, <Data>)?

   • Explicit Relationships: Are connections described explicitly (”X blocks Y”) rather than implied?

   • Schema Adherence: Is domain knowledge presented as structured data, not a dump of raw text?

   5. THE CONSTRAINT AUDIT (Safety & Business Logic)

   • Hard Rules: Are business rules embedded as commands, not suggestions?

   • Tone & Style: Are formatting and voice requirements explicitly defined?

   • Permissions: Is the model explicitly told what it cannot do or access?

   Part 2: The Orchestrator Meta-Prompt

   The Template for High-Fidelity Reasoning

   This is the template used by top-tier AI agents. It forces the LLM to operate within a “structured reasoning cage,” dramatically reducing drift.

   Copy this structure for your system prompt:

[System Instructions] 
You are an AI assistant operating inside a structured context engine. 
Your primary directive is to follow all business rules, domain constraints, and formatting instructions exactly. 
You must not invent facts. You are restricted to the context provided below.

[User Intent]
{inferred_intent}   <-- The interpreted goal, not just the raw text
{explicit_prompt}   <-- What the user actually typed

[Relevant Entities]
{structured_entities} <-- The specific nouns involved (tickets, users, files)

[Relationships]
{entity_relationships} <-- How those nouns connect (dependencies, ownership)

[Session State]
{recent_messages}     <-- Short-term memory
{recent_selections}   <-- Implicit context (clicks, hovers)

[User Profile]
{role}                <-- Who is asking?
{tone_preferences}    <-- How do they like to read?
{prior_examples}      <-- Few-shot examples of success

[Domain Context]
{retrieved_docs}      <-- RAG results, strictly filtered
{summaries}           <-- Distilled knowledge
{attached_metadata}   <-- Environment variables (Time, Location)

[Rules & Constraints]
{business_rules}      <-- Hard logic (e.g., “Never promise dates”)
{policies}            <-- Safety guidelines
{prohibited_actions}  <-- Negative constraints

[Environment Signals]
{calendar_events}     <-- Temporal context
{deadlines}           <-- Urgency signals
{system_status}       <-- Operational reality

[Task Instructions]
Clear, step-by-step instructions for what the model must produce right now.

[Output Schema]
{json_schema_or_output_structure} <-- The exact format required

Adopting this template typically reduces hallucination rates by over 70% in production systems.

Part 3: The Template in Action (Example)

Scenario: The AI Product Management Assistant

To show you exactly how this looks populated, here is a real-world example of an orchestrator prompt generating a Weekly Status Update.

Notice how the “raw” data has been transformed into structured knowledge before the model ever sees it.

THE PROMPT PAYLOAD:

[System Instructions]
You are an AI assistant operating inside a structured context engine for a product team.

You write weekly product status updates for senior stakeholders (VP Product, CTO, CEO) based strictly on the context provided below.

You must:
- Follow all business rules, domain constraints, and formatting instructions exactly.
- Never invent projects, metrics, incidents, or timelines that are not explicitly present in Domain Context.
- Treat the Domain Context and Rules sections as the single source of truth.
- If critical information is missing, state what is missing. Do not guess.

[User Intent]
{inferred_intent}:
“Summarize the most important product changes, progress, risks, and next steps for the past week into an executive-ready weekly update.”
{explicit_prompt}:
“Can you draft this week’s product update for leadership based on what changed since last Monday?”

[Relevant Entities]
{structured_entities}:
- project_roadmap_item:
    id: “PRJ-142”
    title: “Onboarding Funnel Revamp”
    owner: “Sara”
    status: “In Progress”
    target_release: “2025-12-01”
- project_roadmap_item:
    id: “PRJ-087”
    title: “AI Assistant v2”
    owner: “Imran”
    status: “Shipped”
    target_release: “2025-11-15”
- metric:
    id: “MTR-DAU”
    name: “Daily Active Users”
    current_value: 18240
    previous_value: 17680
    unit: “users”
- incident:
    id: “INC-221”
    title: “Checkout Latency Spike”
    status: “Resolved”
    severity: “High”

[Relationships]
{entity_relationships}:
- “PRJ-142” depends_on “PRJ-087”
- “INC-221” impacted “checkout_conversion”
- “MTR-DAU” improved_after “AI Assistant v2” release
- “PRJ-087” linked_to_release “2025.11.15-prod”

[Session State]
{recent_messages}:
- 2025-11-17T09:03Z – User: “Last week’s update is in the doc; I want something similar but shorter.”
- 2025-11-17T09:04Z – Assistant: “Understood, I will keep a similar structure but be more concise.”
- 2025-11-17T09:06Z – User: “Don’t oversell wins; keep it realistic.”

{recent_selections}:
- User highlighted last week’s “Risks & Blockers” section.
- User opened the “AI Assistant v2 – Launch Notes” document.

[User Profile]
{role}:
- “Director of Product, responsible for AI & Growth initiatives.”

{tone_preferences}:
- Confident but not hype.
- Data-informed, not overly narrative.
- Clear separation of “What happened”, “Why it matters”, and “What’s next”.

{prior_examples}:
- Example snippet: “This week we completed the rollout of the new onboarding experiment to 50% of new users. Early results show a +3.2% lift in activation.”

[Domain Context]
{retrieved_docs}:
- “Weekly Update – 2025-11-10” (last week’s product update)
- “AI Assistant v2 – Launch Notes”
- “Onboarding Funnel – Experiment Spec v3”
- “Incident Report – INC-221 Checkout Latency”

{summaries}:
- Last Week’s Update Summary: “Focused on preparing AI Assistant v2 launch, mitigating checkout latency incidents.”
- AI Assistant v2 Launch Notes Summary: “Shipped on 2025-11-15 to 100% of users, goals: improve task completion speed.”
- Incident INC-221 Summary: “High-severity latency issue, resolved within 4 hours, root cause: misconfigured database index.”

{attached_metadata}:
- current_week_range: “2025-11-10 to 2025-11-17”
- timezone: “America/Los_Angeles”
- environment: “Production”

[Rules & Constraints]
{business_rules}:
- Do not share internal incident IDs (e.g., INC-221); describe incidents in business terms.
- Do not reference customers by name; aggregate or anonymize.
- Always tie product work back to business outcomes (activation, retention).

{policies}:
- No forward-looking commitments beyond the existing roadmap.
- Maintain consistency with official metric definitions.

{formatting_requirements}:
- Structure exactly as: 1. Highlights, 2. Metrics & Impact, 3. Risks & Blockers, 4. Next Week.
- Use Markdown headers (H2).
- Total length under 600 words.

{prohibited_actions}:
- Do not fabricate metrics, dates, or launches.
- Do not mention features not explicitly referenced in Domain Context.

[Environment Signals]
{calendar_events}:
- Today is Monday, 2025-11-17.
- “Exec Product Sync” is scheduled for 15:30 today.

{deadlines}:
- Q4 goals lock on 2025-12-01.

{system_status}:
- All systems operational. Analytics data is fresh as of 08:00Z.

[Task Instructions]
Using only the information provided above:
1. Draft a weekly product update that is realistic, grounded, and aligned with the user’s tone.
2. Follow the required structure: Highlights, Metrics & Impact, Risks & Blockers, Next Week.
3. Emphasize what actually changed this week compared to last week.
4. Connect product work to business outcomes using the provided metrics.
5. If there are known gaps (e.g., missing metric results), note them transparently.

[Output Schema]
Return the final result as a JSON object with the following shape:
{
  “highlights_markdown”: “string – Markdown-formatted section”,
  “metrics_and_impact_markdown”: “string – Markdown-formatted section”,
  “risks_and_blockers_markdown”: “string – Markdown-formatted section”,
  “next_week_markdown”: “string – Markdown-formatted section”,
  “notes_for_user”: “string – Any caveats or assumptions made”
}

Final Words: The Age of Context

We are crossing the threshold into what will be remembered as The Age of Context.

In the early 2020s, the industry was obsessed with “Model-First” development. We built wrappers around APIs, relied on clever prompts, and chased demo-day magic. Those tools dazzled for a moment, but they inevitably disappointed the second they encountered the messy reality of enterprise work. Those systems are fading, and they should.

The next decade belongs to Context-First architectures.

We are moving toward systems that do not just generate text, but that understand users with the intuition of a long-time colleague. Systems that navigate institutional history like a veteran employee. Systems that anticipate next steps like a strategic partner. And systems that respect boundaries with the rigor of a compliance officer.

The Only Durable Moat Crucially, this is where the business value lies. A frontier model is a commodity; anyone can rent the smartest brain in the world for a few cents per token. But Context is an asset you own. It is the only defensible moat left.

• Your domain knowledge graph is a moat.

• Your codified business rules are a moat.

• Your user memory bank is a moat.

• Your operational workflows are a moat.

• Your environmental sensors are a moat.

• Your orchestration logic is a moat.

• Your data provenance is a moat.

And these moats compound.

The teams that embrace context engineering today will build products that feel impossibly intelligent—not because they have a better model than their competitors, but because they have architected a system where intelligence is distributed across layers of memory, structure, reasoning, and constraint.

If you are building AI products, this is your invitation—and your responsibility.

Stop waiting for the next model breakthrough to solve your problems.

The true foundation of intelligence is not inference. It is Context.

It’s okay if you don’t know what they are, as this article is primarily intended to explain everything about vector databases in detail.

But given how popular they have become lately, I think it is crucial to be aware of what makes them so powerful that they gained so much popularity, and their practical utility not just in LLMs but in other applications as well.

Let’s dive in!

What are vector databases?

Objective

To begin, we must note that vector databases are NOT new.

In fact, they have existed for a pretty long time now. You have been indirectly interacting with them daily, even before they became widely popular lately. These include applications like recommendation systems, and search engines, for instance.

Simply put, a vector database stores unstructured data (text, images, audio, video, etc.) in the form of vector embeddings.

Each data point, whether a word, a document, an image, or any other entity, is transformed into a numerical vector using ML techniques (which we shall see ahead).

This numerical vector is called an embedding, and the model is trained in such a way that these vectors capture the essential features and characteristics of the underlying data.

Considering word embeddings, for instance, we may discover that in the embedding space, the embeddings of fruits are found close to each other, which cities form another cluster, and so on.

This shows that embeddings can learn the semantic characteristics of entities they represent (provided they are trained appropriately).

Once stored in a vector database, we can retrieve original objects that are similar to the query we wish to run on our unstructured data.

In other words, encoding unstructured data allows us to run many sophisticated operations like similarity search, clustering, and classification over it, which otherwise is difficult with traditional databases.

To exemplify, when an e-commerce website provides recommendations for similar items or searches for a product based on the input query, we’re (in most cases) interacting with vector databases behind the scenes.

Before we get into the technical details, let me give you a couple of intuitive examples to understand vector databases and their immense utility.

Example #1

Let's imagine we have a collection of photographs from various vacations we’ve taken over the years. Each photo captures different scenes, such as beaches, mountains, cities, and forests.

Now, we want to organize these photos in a way that makes it easier to find similar ones quickly.

Traditionally, we might organize them by the date they were taken or the location where they were shot.

However, we can take a more sophisticated approach by encoding them as vectors.

More specifically, instead of relying solely on dates or locations, we could represent each photo as a set of numerical vectors that capture the essence of the image.

💡 While Google Photos doesn't explicitly disclose the exact technical details of its backend systems, I speculate that it uses a vector database to facilitate its image search and organization features, which you may have already used many times.

Let’s say we use an algorithm that converts each photo into a vector based on its color composition, prominent shapes, textures, people, etc.

Each photo is now represented as a point in a multi-dimensional space, where the dimensions correspond to different visual features and elements in the image.

Now, when we want to find similar photos, say, based on our input text query, we encode the text query into a vector and compare it with image vectors.

Photos that match the query are expected to have vectors that are close together in this multi-dimensional space.

Suppose we wish to find images of mountains.

In that case, we can quickly find such photos by querying the vector database for images close to the vector representing the input query.

A point to note here is that a vector database is NOT just a database to keep track of embeddings.

Instead, it maintains both the embeddings and the raw data which generated those embeddings.

Why is that necessary, you may wonder?

Considering the above image retrieval task again, if our vector database is only composed of vectors, we would also need a way to reconstruct the image because that is what the end-user needs.

When a user queries for images of mountains, they would receive a list of vectors representing similar images, but without the actual images.

By storing both the embeddings (the vectors representing the images) and the raw image data, the vector database ensures that when a user queries for similar images, it not only returns the closest matching vectors but also provides access to the original images.

Example #2

In this example, consider an all-text unstructured data, say thousands of news articles, and we wish to search for an answer from that data.

Traditional search methods rely on exact keyword search, which is entirely a brute-force approach and does not consider the inherent complexity of text data.

In other words, languages are incredibly nuanced, and each language provides various ways to express the same idea or ask the same question.

For instance, a simple inquiry like "What's the weather like today?" can be phrased in numerous ways, such as "How's the weather today?", "Is it sunny outside?", or "What are the current weather conditions?".

This linguistic diversity makes traditional keyword-based search methods inadequate.

As you may have already guessed, representing this data as vectors can be pretty helpful in this situation too.

Instead of relying solely on keywords and following a brute-force search, we can first represent text data in a high-dimensional vector space and store them in a vector database.

When users pose queries, the vector database can compare the vector representation of the query with that of the text data, even if they don't share the exact same wording.

How to generate embeddings?

At this point, if you are wondering how do we even transform words (strings) into vectors (a list of numbers), let me explain.

To build models for language-oriented tasks, it is crucial to generate numerical representations (or vectors) for words.

This allows words to be processed and manipulated mathematically and perform various computational operations on words.

The objective of embeddings is to capture semantic and syntactic relationships between words. This helps machines understand and reason about language more effectively.

In the pre-Transformers era, this was primarily done using pre-trained static embeddings.

Essentially, someone would train embeddings on, say, 100k, or 200k common words using deep learning techniques and open-source them.

Consequently, other researchers would utilize those embeddings in their projects.

The most popular models at that time (around 2013-2017) were:

• Glove

• Word2Vec

• FastText, etc.

These embeddings genuinely showed some promising results in learning the relationships between words.

For instance, at that time, an experiment showed that the vector operation (King - Man) + Woman returned a vector near the word Queen.

That’s pretty interesting, isn’t it?

In fact, the following relationships were also found to be true:

• Paris - France + Italy ≈ Rome

• Summer - Hot + Cold ≈ Winter

• Actor - Man + Woman ≈ Actress

• and more.

So, while these embeddings captured relative word representations, there was a major limitation.

Consider the following two sentences:

• Convert this data into a table in Excel.

• Put this bottle on the table.

Here, the word “table” conveys two entirely different meanings:

• The first sentence refers to a “data” specific sense of the word “table.”

• The second sentence refers to a “furniture” specific sense of the word “table.”

Yet, static embedding models assigned them the same representation.

Thus, these embeddings didn’t consider that a word may have different usages in different contexts.

But this was addressed in the Transformer era, which resulted in contextualized embedding models powered by Transformers, such as:

• BERT: A language model trained using two techniques:

  • Masked Language Modeling (MLM): Predict a missing word in the sentence, given the surrounding words.

  • Next Sentence Prediction (NSP).

  • We shall discuss it in a bit more detail shortly.

• DistilBERT: A simple, effective, and lighter version of BERT, which is around 40% smaller:

  • Utilizes a common machine learning strategy called student-teacher theory.

  • Here, the student is the distilled version of BERT, and the teacher is the original BERT model.

  • The student model is supposed to replicate the teacher model’s behavior.

• SentenceTransformer: If you read the most recent deep dive on building classification models on ordinal data, we discussed this model there.

  • Essentially, the SentenceTransformer model takes an entire sentence and generates an embedding for that sentence.

• This differs from the BERT and DistilBERT models, which produce an embedding for all words in the sentence.

There are more models, but we won't go into more detail here, and I hope you get the point.

The idea is that these models are quite capable of generating context-aware representations, thanks to their self-attention mechanism and appropriate training mechanism.

BERT

For instance, if we consider BERT again, we discussed above that it uses the masked language modeling (MLM) technique and next sentence prediction (NSP).

These steps are also called the pre-training step of BERT because they involve training the model on a large corpus of text data before fine-tuning it on specific downstream tasks.

Pre-training, in the context of machine learning model training, refers to the initial phase of training where the model learns general language representations from a large corpus of text data. The goal of pre-training is to enable the model to capture the syntactic and semantic properties of language, such as grammar, context, and relationships between words. While the text itself is unlabeled, MLM and NSP are two tasks that help us train the model in a supervised fashion. Once the model is trained, we can use the language understanding capabilities that the model acquired from the pre-training phase, and fine-tune the model on task-specific data.

Moving on, let’s see how the pre-training objectives of masked language modeling (MLM) and next sentence prediction (NSP) help BERT generate embeddings.

#1) Masked Language Modeling (MLM)

• In MLM, BERT is trained to predict missing words in a sentence. To do this, a certain percentage of words in most (not all) sentences are randomly replaced with a special token, [MASK].

• BERT then processes the masked sentence bidirectionally, meaning it considers both the left and right context of each masked word, that is why the name “Bidirectional Encoder Representation from Transformers (BERT).”

• For each masked word, BERT predicts what the original word is supposed to be from its context. It does this by assigning a probability distribution over the entire vocabulary and selecting the word with the highest probability as the predicted word.

• During training, BERT is optimized to minimize the difference between the predicted words and the actual masked words, using techniques like cross-entropy loss.

#2) Next Sentence Prediction (NSP)

• In NSP, BERT is trained to determine whether two input sentences appear consecutively in a document or whether they are randomly paired sentences from different documents.

• During training, BERT receives pairs of sentences as input. Half of these pairs are consecutive sentences from the same document (positive examples), and the other half are randomly paired sentences from different documents (negative examples).

• BERT then learns to predict whether the second sentence follows the first sentence in the original document (label 1) or whether it is a randomly paired sentence (label 0).

• Similar to MLM, BERT is optimized to minimize the difference between the predicted labels and the actual labels, using techniques like binary cross-entropy loss.

If we look back to MLM and NSP, in both cases, we did not need a labeled dataset to begin with. Instead, we used the structure of the text itself to create the training examples. This allows us to leverage large amounts of unlabeled text data, which is often more readily available than labeled data.

Now, let's see how these pre-training objectives help BERT generate embeddings:

• MLM: By predicting masked words based on their context, BERT learns to capture the meaning and context of each word in a sentence. The embeddings generated by BERT reflect not just the individual meanings of words but also their relationships with surrounding words in the sentence.

• NSP: By determining whether sentences are consecutive or not, BERT learns to understand the relationship between different sentences in a document. This helps BERT generate embeddings that capture not just the meaning of individual sentences but also the broader context of a document or text passage.

With consistent training, the model learns how different words relate to each other in sentences. It learns which words often come together and how they fit into the overall meaning of a sentence.

This learning process helps BERT create embeddings for words and sentences, which are contextualized, unlike earlier embeddings like Glove and Word2Vec:

Contextualized means that the embedding model can dynamically generate embeddings for a word based on the context they were used in.

As a result, if a word would appear in a different context, the model would return a different representation.

This is precisely depicted in the image below for different uses of the word Bank.

For visualization purposes, the embeddings have been projected into 2d space using t-SNE.

As depicted above, the static embedding models — Glove and Word2Vec produce the same embedding for different usages of a word.

However, contextualized embedding models don’t.

In fact, contextualized embeddings understand the different meanings/senses of the word “Bank”:

• A financial institution

• Sloping land

• A Long Ridge, and more.

As a result, these contextualized embedding models address the major limitations of static embedding models.

The point of the above discussion is that modern embedding models are quite proficient at the encoding task.

As a result, they can easily transform documents, paragraphs, or sentences into a numerical vector that captures its semantic meaning and context.

In an era of voice-driven interfaces and AI-powered assistants, extracting meaningful insights from audio is more important than ever. Whether you're a developer, product builder, or AI tinkerer, this guide will show you how to build a full-fledged Audio Analysis Toolkit that can:

• Transcribe audio with speaker labels and timestamps

• Summarize long conversations

• Detect sentiment and emotional tone

• Extract key topics

• Enable natural-language Q&A on audio content

The best part? You can use it either as:

1. A beautiful web UI powered by Streamlit, OR

2. A backend MCP server connected to Claude via Cursor for agentic interaction.

Let’s dive into both modes of using the toolkit.

Tech Stack Overview

• AssemblyAI – API for audio transcription and advanced analysis (sentiment, topics, summarization)

• Streamlit – Rapid frontend for building interactive UI

• Cursor MCP (Model Context Protocol) – Interface for agents like Claude to call backend functions

• Claude (via Cursor) – Agent that can consume your toolkit through natural language

• Python 3.12+

Here's our workflow:

• User's audio input is sent to AssemblyAI via a local MCP server.

• ​AssemblyAI​ transcribes it while providing the summary, speaker labels, sentiment, and topics.

• Post-transcription, the user can also chat with audio.

Dual Tool Structure: Transcription + Audio Analysis (via MCP)

Transcription MCP tool

This tool accepts an audio input from the user and transcribes it using AssemblyAI.

We also store the full transcript to use in the next tool.

Audio analysis tool

Next, we have a tool that returns specific insights from the transcript, like speaker labels, sentiment, topics, and summary.

Based on the user’s input query, the corresponding flags will be automatically set to True when the Agent will prepare the tool call via MCP:

Bonus: It’s RAG-Ready

Once transcribed, the transcript can even be indexed into a vector database or used in an LLM context window for true RAG-style audio Q&A, especially useful for:

• Meeting assistants

• Interview breakdowns

• Legal depositions

• Customer call analysis

Let’s implement this!

Option 1: Use as a Streamlit Web App 

This mode is perfect if you want a user-friendly, visually rich interface for uploading and analyzing audio files.

This web-based interface is powered by Streamlit and comes with:

• Audio file upload: .mp3, .wav, .mp4, etc.

• Full transcript with timestamps

• Speaker-wise breakdown

• Sentiment visualized with emojis and colors

• AI-generated summaries

• Top topics and relevance

• Ask questions about the audio in a chatbot-style interface

How to Run It? 

1. Clone the repo and set up the environment.

git clone https://github.com/edityeah/ai-hub.git
cd audio-analysis-toolkit
python -m venv .venv
source .venv/bin/activate    # Or .venvScriptsactivate on Windows
pip install -r requirements.txt	

2. Add your AssemblyAI API Key

Create a .env file:

ASSEMBLYAI_API_KEY=your_key_here		

Run the app

streamlit run app.py		

Option 2: Use as a Cursor MCP Server (for Claude Agents)

If you prefer working agentically — that is, using an AI like Claude to invoke capabilities — you can run this toolkit as an MCP server.

What is MCP?

MCP (Model Context Protocol) is how tools can plug into Cursor and be used by agents like Claude. It allows your local Python scripts to respond to natural language instructions inside the Cursor IDE.

Setting It Up:

1. Ensure server.py is in your project directory (this script exposes transcription logic via HTTP or CLI).

2. Configure Cursor MCP:
   In Cursor → Preferences → MCP Servers → Global, add:

{
  "mcpServers": {
    "assemblyai": {
      "command": "python",
      "args": ["server.py"],
      "env": {
        "ASSEMBLYAI_API_KEY": "your_key_here"
      }
    }
  }
}			

3. Open the project folder in Cursor
Cursor will automatically launch the MCP server if the config and working directory are correct.

4. Call it from Claude:
Inside Cursor, invoke the tool like this:

@assemblyai: transcribe "path/to/audio.mp3"		

Claude will now interact with your audio analysis server in real-time and return intelligent answers.

Final Thoughts

With just a few lines of Python and AssemblyAI’s APIs, you can build a full-featured audio intelligence system. Whether you want to offer this as a product, internal tool, or agentic function inside your AI IDE, this toolkit gives you the flexibility to do both.

💡 Ready to build? Try both modes and see what fits your workflow best!

In today's organizations, data is scattered across countless platforms—Slack, Gmail, SQL databases, Google Drive, GitHub, and more. Bringing these data sources into one unified, queryable system is often complex, requiring expensive data integration platforms or risky cloud-based services.

I've solved this by building a 100% local MCP (Model Context Protocol) server that can connect to 200+ data sources and enable natural language querying across them via a simple chat interface.

Let’s walk you through how I've built this powerful federated query system using MindsDB, Cursor IDE, and Docker.

The Unified MCP Server

This project revolves around building a local-first, modular, and scalable MCP server that:

• Connects to over 200+ data sources.

• Allows natural language querying across structured and unstructured platforms.

• Is completely self-hosted, ensuring maximum data privacy and security.

I've used:

• MindsDB: A federated query engine that understands complex, multi-source queries.

• Cursor IDE: The interface that hosts the MCP chat and connects to the server.

• Docker: For local containerization and seamless deployment.

Key Features

FeatureDescription🔍 Multi-source QueryingQuery 200+ data sources with a single natural language prompt.🧠 Context-Aware AIAI responses tailored to your connected organizational data.💬 Real-time Chat InterfaceUnified chat window powered by Cursor IDE.🔧 Modular DesignEasily plug in new data sources via MindsDB integrations.🐳 Docker-based DeploymentFully local deployment ensuring security and control.🔒 Privacy-firstData remains on your local machines—no cloud dependency.

How It Works: Architecture Flow

1. User submits a query via Cursor’s chat interface.

2. Cursor client forwards the query to the MCP server.

3. MCP server sends the query to MindsDB’s federated query engine.

4. MindsDB retrieves results from the relevant connected data sources.

5. MCP server receives results and passes them back to Cursor.

6. Cursor displays the AI-generated response in the chat interface.

Step-by-Step Implementation Details

Prerequisites:

• Docker

• Cursor IDE

• MindsDB (installed locally)

Docker Setup

MindsDB provides a pre-built Docker image for quick setup.

To install MindsDB locally, I ran the following command:

docker run --name mindsdb_container -e MINDSDB_APIS="http,mcp" 
-p 47334:47334 -p 47337:47337 mindsdb/mindsdb		

🖥️ Start MindsDB GUI

After installing, go to  127.0.0.1:47334 in your browser to access the MindsDB editor.

This GUI allows me to:

• Connect over 200+ data sources

• Run SQL queries against them

• Manage the federated query engine visually

Integrate Data Sources

Let's start building our federated query engine by connecting our data sources to MindsDB.

We use Slack, Gmail, GitHub, and Hacker News as our federated data sources.

Integrate MCP Server with Cursor

After building the federated query engine, let's unify our data sources by connecting them to MindsDB's MCP server.

Go to: File → Preferences → Cursor Settings → MCP → Add new global MCP server

In the JSON file, add the following 👇

Done! Our MindsDB MCP server is live and connected to Cursor!

The MCP server offers two tools:

• list_databases: Lists all data sources connected to MindsDB.

• query: Answers user queries on the federated data.

Apart from Claude and Cursor, the MindsDB MCP server also works with the new OpenAI MCP integration:

The usage of list_databases is depicted below. We asked it to list all the data sources it can access and retrieved the four data sources we connected earlier:

Below is another usage, where we asked it to list the available Slack channels.

While this is quite basic, I have shown a much more sophisticated usage in the video attached at the top.

• I fetched info from Hacker News.

• I generated a nicely formatted summary of the fetched details and sent that as a message to Slack.

Chatting With My Data

I can now submit queries directly via Cursor’s chat interface:

• “Show me the last five emails from Gmail.”

• “Summarize recent Slack mentions.”

• “Fetch the top stories from Hacker News.”

• “List all open GitHub issues in my repository.”

Why I Built This

• Data Privacy: Fully local, no cloud dependency.

• Unified Access: Query Slack, Gmail, GitHub, and more from one place.

• Scalable: Easy to add new data sources.

• Simple Interface: Natural language queries—no SQL skills required.

💡 Closing Thoughts

I built this unified MCP server to make data querying seamless, private, and conversational.

It’s:

• Local-first for security.

• Modular for flexibility.

• Simple for everyday use.

If you’re looking to own your data workflow, this is a powerful, scalable approach.

Lately, there’s been a lot of buzz around Model Context Protocol (MCP). You’ve probably heard of it.

Today, let’s break it down.

Intuitively speaking, MCP is like a USB-C port for your AI applications.

Just as USB-C offers a standardized way to connect devices to various accessories, MCP standardizes how AI apps connect to different data sources and tools.

Let’s dive in a bit more technically.

What is MCP?

At its core, MCP follows a client-server architecture, where a host application can connect to multiple servers.

It has three key components:

• Host

• Client

• Server

Here’s a quick overview before we go deeper 👇

• Host: Represents any AI app (like Claude Desktop or Cursor) that provides an environment for AI interactions, accesses tools and data, and runs the MCP Client.

• MCP Client: Operates within the host and enables communication with MCP servers.

• MCP Server: Exposes specific capabilities and provides access to data like:

  • Tools: Let LLMs perform actions via your server

  • Resources: Share data/content from your server with LLMs

  • Prompts: Provide reusable prompt templates and workflows

How Does the Client-Server Communication Work?

Understanding this exchange is key to building your own MCP client-server setup.

Let’s look at the flow:

1. Capability Exchange:

   • The client sends an initial request to learn the server’s capabilities.

   • The server responds with a list of its tools, resources, prompt templates, and other features.

   📦 Example: A Weather API server may reply with the tools it offers (like “getTemperature”), available prompt workflows, and supported parameters.

2. Connection Acknowledgment:

   • The client confirms successful connection.

   • All further interactions build on this shared understanding.

Why Is This Powerful?

Here’s a common pain point with traditional APIs:

• If your API initially requires two parameters (e.g., location and date for a weather service), users integrate their applications to send requests with those exact parameters.

• Later, if you decide to add a third required parameter (e.g., unit for temperature units like Celsius or Fahrenheit), the API’s contract changes.

• This means all users of your API must update their code to include the new parameter. If they don’t update, their requests might fail, return errors, or provide incomplete results.

How MCP Solves This?

MCP introduces a dynamic and flexible approach that contrasts sharply with traditional APIs.

• For instance, when a client (e.g., an AI application like Claude Desktop) connects to an MCP server (e.g., your weather service), it sends an initial request to learn the server’s capabilities.

• The server responds with details about its available tools, resources, prompts, and parameters. For example, if your weather API initially supports location and date, the server communicates these as part of its capabilities.

• If you later add a unit parameter, the MCP server can dynamically update its capability description during the next exchange. The client doesn’t need to hardcode or predefine the parameters—it simply queries the server’s current capabilities and adapts accordingly.

• This way, the client can then adjust its behavior on-the-fly, using the updated capabilities (e.g., including unit in its requests) without needing to rewrite or redeploy code.

Wrapping Up

I hope this clarifies what MCP does and how it compares to traditional APIs.

Do you think MCP is more powerful than traditional API setups?

In upcoming posts, we’ll explore how to build custom MCP servers and share hands-on demos.

Stay tuned!

In a world where data is the new oil, querying it shouldn't feel like learning a new language.

And yet, that’s what most non-technical users face when trying to interact with databases.

Enter Vanna AI: a powerful, 100% open-source Retrieval-Augmented Generation (RAG) framework that converts natural language into accurate SQL queries—securely, scalably, and seamlessly.

With over 14,000 GitHub stars, Vanna has captured the attention of data teams, product managers, and developers for one simple reason:

It just works.

https://youtu.be/hhWId32mpyI

🚀 What is Vanna AI?

Vanna AI is an open-source Text-to-SQL engine built on RAG (Retrieval-Augmented Generation), specifically designed to handle complex, dynamic datasets like the ones used in modern enterprises.

At its core, Vanna makes databases talk back in plain English. Whether you're working with Snowflake, Redshift, Postgres, or BigQuery, Vanna translates your questions into SQL queries that run directly on your data.

🧠 How Vanna Works in 2 Steps

Using Vanna is surprisingly simple:

1️⃣ Train a RAG model on your database schema, documentation, and query logs (it creates a knowledge base).

2️⃣ Ask your business question in plain English. Vanna generates SQL that can directly run on your DB—or be reviewed by analysts before execution.

And yes, it gets smarter with each question.

🔑 Key Features

FeatureDescription🎯 High AccuracyHandles nested, complex SQL queries with better performance than most LLM-only tools.🧠 Self-LearningContinuously improves by learning from past queries and feedback.🔒 Secure by DesignYour data stays within your environment—Vanna only needs metadata, not actual records.🌐 Database-AgnosticSupports all major SQL databases: Postgres, Snowflake, Redshift, MySQL, SQL Server, and more.💻 Flexible FrontendsIntegrate Vanna with Jupyter Notebooks, Slack bots, Streamlit apps, or even Flask APIs.

⚙️ Under the Hood: Architecture Overview

Vanna combines the best of three layers:

1. LLMs (OpenAI, Anthropic, Gemini, etc.) – For interpreting natural language and generating human-like SQL.

2. Vector DBs (ChromaDB, FAISS, Qdrant, etc.) – To search and retrieve relevant metadata, schemas, or logs.

3. SQL Connectors – For seamless execution or validation on live databases.

The stack is modular—you choose which LLM, which vector store, and which database you want to plug in.

💡 Use Cases

Vanna isn’t just a technical marvel. It’s also practically useful:

• 📊 Business Intelligence: Non-technical users can query without writing SQL.

• 🔍 Exploratory Analysis: Analysts can speed up data discovery.

• 👨‍🏫 Education: Helps learners understand the SQL equivalent of business questions.

• 🛠 Internal Tools: Integrate into Slack, dashboards, or internal portals for real-time data Q&A.

🔧 Installation & Example

Want to try it? Just:

pip install vanna

Note: Depending on the LLM and vector store used, additional packages may be required.

Then Basic Setup (OpenAI + ChromaDB Example)

from vanna.openai.openai_chat import OpenAI_Chat
from vanna.chromadb.chromadb_vector import ChromaDB_VectorStore
class MyVanna(ChromaDB_VectorStore, OpenAI_Chat):
    def __init__(self, config=None):
        ChromaDB_VectorStore.__init__(self, config=config)
        OpenAI_Chat.__init__(self, config=config)
vn = MyVanna(config={'api_key': 'your-openai-key', 'model': 'gpt-4'})
vn.train(schema='your_schema', logs='your_query_logs')
vn.ask("What are the top 5 products by revenue?")

🎓 Training Vanna AI

You can improve Vanna's output quality by training it on your:

✍️ DDL Statements

vn.train(ddl="""
    CREATE TABLE IF NOT EXISTS my_table (
        id INT PRIMARY KEY,
        name VARCHAR(100),
        age INT
    )
""")

📚 Business Documentation

vn.train(documentation="Our business defines XYZ as ...")

📈 SQL Query Examples

vn.train(sql="SELECT name, age FROM my_table WHERE name = 'John Doe'")		

💬 Ask Questions

vn.ask("What are the top 10 customers by sales?")	

Returns:

SELECT c.c_name AS customer_name,
       SUM(l.l_extendedprice * (1 - l.l_discount)) AS total_sales
FROM   lineitem l
JOIN   orders o ON l.l_orderkey = o.o_orderkey
JOIN   customer c ON o.o_custkey = c.c_custkey
GROUP BY customer_name
ORDER BY total_sales DESC
LIMIT 10;

And if connected to a DB, you also get a result table and a Plotly chart.

RAG vs Fine-Tuning

RAG ApproachFine-TuningEasily replace LLMsHardcoded to one modelCheap to runExpensive to train & useEasily editable knowledge baseHard to remove outdated training data

Why Vanna AI?

• ✅ Accuracy-first: Improves with more training data

• 🔒 Data secure: SQL execution & training remain local

• 🥇 Customizable: Choose your own stack (LLM + Vector Store)

• 🌟 Ready for production: Jupyter for devs, Streamlit or Slack for users

Interfaces You Can Build With

• Jupyter notebooks (great for data analysts)

• Streamlit apps (dashboard-ready)

• Slack bots (team-friendly)

• Flask APIs (for integration into internal tools)

• Chainlit (chat-style data assistants)

What It Doesn’t Do (Yet)

• Data modification (INSERT/UPDATE) is not the focus.

• Not optimized for non-SQL data sources (e.g., NoSQL).

• The full install can be bulky (~400MB)—but it’s modular.

Final Thoughts

Tools like Vanna AI represent a shift in how we think about querying data. Instead of expecting everyone to learn SQL, Vanna brings the query layer to where the user is—in plain language.

And because it’s open-source and modular, you’re in full control. No vendor lock-ins. No cloud-only limitations. Just powerful, customizable NLP-to-SQL magic.

📚 Learn More & Contribute

Explore the full documentation and source code on the official GitHub repo.

Have a use case in mind? Fork it. Tweak it. Or just start asking questions—and let your data speak.

Imagine having a voice assistant that doesn't just respond from predefined answers but actually retrieves relevant context from your own local documents and responds using your own cloned voice.

Welcome to the era of real-time voice-based Retrieval-Augmented Generation (RAG) agents!

In this post, I’ll walk you through:

• How I built a real-time voice-based RAG agent

• How I integrated AssemblyAI for transcription

• How the agent uses LiveKit for voice streaming

• How it responds in my cloned voice using Cartesia AI

• Practical real-life use cases that make this more than a fun demo

⚙️ Architecture Overview

Here’s what’s happening under the hood:

Technologies Used:

• 🧠 LLM: Google’s Gemma 3 via LlamaIndex

• 📄 Indexing: Local PDFs indexed with LlamaIndex

• 🗣️ STT: AssemblyAI

• 🧏‍♂️ TTS: Cartesia AI

• 🎤 VAD: Silero

• 📡 Orchestration: LiveKit agents playground

🛠️ Building the Agent

For those interested in the architecture, you can check out the open-source code here:

GitHub

Or follow the instructions below:

1. Set up your environment

pip install -r requirements.txt						

Create a .env file with:

OPENAI_API_KEY=your_openai_api_key
CARTESIA_API_KEY=your_cartesia_api_key
LIVEKIT_URL=your_livekit_url
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret
ASSEMBLYAI_API_KEY=your_assemblyai_api_key			

2. Index your documents

Place your PDFs in a docs/ folder and run:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
reader = SimpleDirectoryReader("docs")
docs = reader.load_data()
index = VectorStoreIndex.from_documents(docs)
index.storage_context.persist(persist_dir="chat-engine-storage")

3. Configure your agent (Python)

Use the VoicePipelineAgent from LiveKit. Key configurations:

agent = VoicePipelineAgent(
    vad=silero.VAD.load(),
    stt=assemblyai.STT(),
    llm=llama_index.LLM(chat_engine=chat_engine),
    tts=cartesia.TTS(
        model="sonic-2",
        voice="your-cloned-voice-id",
    ),
    chat_ctx=chat_context,
)		

4. Run the agent

python voice_agent.py start		

Now head to agents-playground.livekit.io, and click Connect. Speak your question out loud—something relevant to the documents you indexed in Step 2.

🧠 The agent will detect your voice, transcribe it using AssemblyAI, fetch context from your documents, generate a response using Gemma 3, and speak it back using Cartesia's voice synthesis engine.

5: Clone Your Voice with Cartesia

By default, your agent uses a robotic voice provided by Cartesia. But you can replace that with your own cloned voice.

1. Visit Cartesia.ai

2. Click on Instant Voice Clone

3. Provide a name and a short 5-second audio sample (e.g., "Hey, this is Aditya and this is my voice for cloning")

4. Cartesia will return a voice_id

5. Replace the default voice ID in your .env file:

CARTESIA_VOICE_ID=your_cloned_voice_id			

6. Restart your agent to use your own voice!

Below is the full Python code I used to build this agent:

import logging
import os
from dotenv import load_dotenv
from livekit.agents import JobContext, JobProcess, WorkerOptions, cli
from livekit.agents.job import AutoSubscribe
from livekit.agents.llm import (
    ChatContext,
)
from livekit.agents.pipeline import VoicePipelineAgent
from livekit.plugins import cartesia, silero, llama_index, assemblyai
load_dotenv()
logger = logging.getLogger("voice-assistant")
from llama_index.llms.ollama import Ollama
from llama_index.core import (
    SimpleDirectoryReader,
    StorageContext,
    VectorStoreIndex,
    load_index_from_storage,
    Settings
)
from llama_index.core.chat_engine.types import ChatMode
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
load_dotenv()
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
llm=Ollama(model="gemma3", request_timeout=120.0)
Settings.llm = llm
Settings.embed_model = embed_model
# check if storage already exists
PERSIST_DIR = "./chat-engine-storage"
if not os.path.exists(PERSIST_DIR):
    # load the documents and create the index
    documents = SimpleDirectoryReader("docs").load_data()
    index = VectorStoreIndex.from_documents(documents)
    # store it for later
    index.storage_context.persist(persist_dir=PERSIST_DIR)
else:
    # load the existing index
    storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
    index = load_index_from_storage(storage_context)
def prewarm(proc: JobProcess):
    proc.userdata["vad"] = silero.VAD.load()
async def entrypoint(ctx: JobContext):
    chat_context = ChatContext().append(
        role="system",
        text=(
            "You are a funny, witty assistant."
            "Respond with short and concise answers. Avoid using unpronouncable punctuation or emojis."
        ),
    )
    chat_engine = index.as_chat_engine(chat_mode=ChatMode.CONTEXT)
    logger.info(f"Connecting to room {ctx.room.name}")
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
    participant = await ctx.wait_for_participant()
    logger.info(f"Starting voice assistant for participant {participant.identity}")
    stt_impl = assemblyai.STT()
    agent = VoicePipelineAgent(
        vad=ctx.proc.userdata["vad"],
        stt=stt_impl,
        llm=llama_index.LLM(chat_engine=chat_engine),
        tts=cartesia.TTS(
            model="sonic-2",
            voice="794f9389-aac1-45b6-b726-9d9369183238",
        ),
        chat_ctx=chat_context,
    )
    agent.start(ctx.room, participant)
    await agent.say(
        "Hey there! How can I help you today?",
        allow_interruptions=True,
    )
if __name__ == "__main__":
    print("Starting voice agent...")
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
        ),
    )
				
			

🧠 Real-Life Use Cases

Here’s where this gets seriously powerful:

1. Education & Research

Ask research papers questions by simply speaking. No UI. No ChatGPT. Just voice.

2. Enterprise Knowledge Assistants

Replace dumb IVRs. Let users talk to internal wikis, HR docs, SOPs.

3. Elderly or Visually Impaired Users

Make documentation searchable and explorable without screens.

4. Developer Agents

Let engineers talk to API docs, CLI tools, changelogs—all via voice.

5. Personalized Voice Agents

Build your own Jarvis-style assistant trained on your workflows.

Stay tuned for more experiments in real-time AI agents 🚀

Optical Character Recognition (OCR) is an essential tool for extracting text from images, making it easier to digitize documents, analyze handwritten notes, and process structured information. With advancements in AI, OCR has become more accessible and efficient, allowing developers and product managers to integrate it into their workflows seamlessly.

In this guide, we’ll walk you through building your own locally running OCR app using Google’s Gemma-3 Vision model, Ollama, and Streamlit. This app allows users to upload an image and extract structured text in Markdown format, ensuring clarity and organization.

What is Ollama, Gemma-3, and Streamlit?

Before we dive into the setup, let’s understand the key components of our application:

• Ollama: A lightweight tool that allows you to run large language models locally on your machine. It helps deploy models efficiently without requiring cloud-based services.

• Gemma-3 Vision Model: A powerful multimodal AI model developed by Google that can process both images and text, making it ideal for OCR tasks.

• Streamlit: A Python-based framework that makes it easy to create interactive web applications for machine learning models and data visualization.

With these three tools, we can build a fully functional OCR app that runs entirely on your local machine.

Step-by-Step Guide to Set Up and Run the OCR App

Github

1. Install Ollama

To run Gemma-3 locally, we first need to install Ollama.

On Linux:

curl -fsSL https://ollama.com/install.sh | sh

On Mac:

brew install ollama

On Windows (using WSL 2):

wsl --install
curl -fsSL https://ollama.com/install.sh | sh			

After installation, verify that Ollama is installed by running:

ollama --version			

2. Download the Gemma-3 Vision Model

Once Ollama is set up, download and load the Gemma-3 Vision model:

ollama run gemma3:12b

This command ensures that the model is available on your local machine for inference.

3. Install Python and Required Dependencies

Ensure you have Python 3.11 or later installed. Then, install the necessary Python packages:

pip install streamlit ollama pillow	

4. Create the Streamlit App

Now, let’s write the Python script that powers our OCR application. Save the following code as app.py:

import streamlit as st
import ollama
from PIL import Image
import io
import base64
# Page configuration
st.set_page_config(
    page_title="Gemma-3 OCR",
    page_icon="🔎",
    layout="wide",
    initial_sidebar_state="expanded"
)
# Title and description in main area
st.markdown("""
    # <img src="data:image/png;base64,{}" width="50" style="vertical-align: -12px;"> Gemma-3 OCR
""".format(base64.b64encode(open("./assets/gemma3.png", "rb").read()).decode()), unsafe_allow_html=True)
# Add clear button to top right
col1, col2 = st.columns([6,1])
with col2:
    if st.button("Clear 🗑️"):
        if 'ocr_result' in st.session_state:
            del st.session_state['ocr_result']
        st.rerun()
st.markdown('<p style="margin-top: -20px;">Extract structured text from images using Gemma-3 Vision!</p>', unsafe_allow_html=True)
st.markdown("---")
# Move upload controls to sidebar
with st.sidebar:
    st.header("Upload Image")
    uploaded_file = st.file_uploader("Choose an image...", type=['png', 'jpg', 'jpeg'])
    if uploaded_file is not None:
        # Display the uploaded image
        image = Image.open(uploaded_file)
        st.image(image, caption="Uploaded Image")
        if st.button("Extract Text 🔍", type="primary"):
            with st.spinner("Processing image..."):
                try:
                    response = ollama.chat(
                        model='gemma3:12b',
                        messages=[{
                            'role': 'user',
                            'content': """Analyze the text in the provided image. Extract all readable content
                                        and present it in a structured Markdown format that is clear, concise, 
                                        and well-organized. Ensure proper formatting (e.g., headings, lists, or
                                        code blocks) as necessary to represent the content effectively.""",
                            'images': [uploaded_file.getvalue()]
                        }]
                    )
                    st.session_state['ocr_result'] = response.message.content
                except Exception as e:
                    st.error(f"Error processing image: {str(e)}")
# Main content area for results
if 'ocr_result' in st.session_state:
    st.markdown(st.session_state['ocr_result'])
else:
    st.info("Upload an image and click 'Extract Text' to see the results here.")
# Footer
st.markdown("---")
st.markdown("Made with ❤️ using Gemma-3 Vision Model | [Adityeah](https://adityeah.in/)")
				
			

5. Run the Streamlit App

Once the script is ready, launch the Streamlit app by running:

streamlit run app.py

Your OCR app will be accessible in your browser, allowing you to upload an image and extract structured text.

Use Cases of OCR Across Domains

Education

1. Automated Exam Grading: Teachers can upload images of students' answer sheets, and the model can compare student responses with predefined correct answers to automate grading and feedback generation.

2. Homework Evaluation: Students can submit handwritten homework, which the model can analyze for correctness and provide instant feedback.

3. Doubt Resolution: Students can upload handwritten queries, and the model can extract and structure the content for quick reference and resolution.

4. Digitizing Study Materials: OCR can convert handwritten notes, books, and lecture slides into digital formats for better accessibility.

Finance

1. Automated Invoice Processing: OCR extracts and processes financial data from invoices and receipts for accounting automation.

2. Check and Bank Statement Scanning: Quickly digitize paper checks and bank statements to track transactions more efficiently.

3. Fraud Detection: OCR can verify documents for discrepancies in financial reports and detect potential fraud.

Healthcare

1. Medical Record Digitization: Convert handwritten doctor notes and prescriptions into electronic health records (EHRs) for better accessibility.

2. Insurance Claims Processing: Extract data from claim forms and automate insurance approvals.

3. Patient Form Processing: Reduce paperwork by digitizing patient intake forms and medical histories.

Retail & Logistics

1. Barcode and Label Scanning: Automate inventory tracking by extracting data from labels and barcodes.

2. Automated Receipt Processing: Digitize and categorize receipts for expense tracking and reimbursement.

3. Order Management: OCR extracts shipping and order details from printed documents to streamline logistics operations.

With OCR technology becoming more powerful and accessible, businesses across industries can leverage it to automate workflows, enhance efficiency, and improve data accuracy.

This post will transform how you build products, come up with new ideas, and operate as a PM.

Whether you’re already deep into AI tools or just getting started, you’ll learn what tools you should be paying attention to, which tool to use when, and how to get unstuck when you run into an issue. You’ll find a collection of battle-tested prompts, real-world examples, and a step-by-step guide you can put into practice immediately. Imagine being able to turn Figma designs into a working app with a few clicks, or turn your PRD into a working prototype in minutes. This is all possible, and you’ll learn how.

If you haven’t been paying close attention over the past six months, you may have missed the rise of tools like Cursor, Replit Agent, v0, Bolt, and other new cutting-edge AI tools that allow you to build working apps in minutes. For example, it took me 10 minutes to build this 2-D tank game (with an AI opponent included), merely using this series of prompts:

• “Build a 2d tank game with an AI opponent.”

• “Add collision for the shot when it hits a tank.”

• “When health hits zero, play an animation and reset the game.”

• “Improve the acceleration for player movement.”

• “Make it so holding down the space bar has a timer to shoot a 2nd time.”

• “Add power ups to the map.”

Pretty cool. But what’s cooler is that you can use these tools to build functional prototypes from a Figma design, convert a rough hand-drawn sketch to a working app, translate a PRD document into an interactive prototype, or even build a usable internal tool for your team, with no coding ability. In this post, I’ll cover the basics of AI prototyping, show how to get good results out of the most popular tools, and walk through an end-to-end example of building a prototype in less than 10 minutes.

Choosing your tooling

Current AI development tools come in three types:

1. Chatbots (e.g. Claude, ChatGPT): The AI tools you probably know, which can also write and explain basic code

2. Cloud development environments (e.g. Replit, Bolt, v0, Lovable): Full-stack platforms that can build and run your apps in the cloud

3. Local developer assistants (e.g. GitHub Copliot, Cursor, Windsurf, Zed): Development environments (i.e. IDEs) that help you write code with the help of AI

Let’s review the most popular tools in each category to see what they can do and what we can build.

Chatbots (ChatGPT, Claude)

Best for: Prototypes that are just one page and don’t have complex design requirements, like calculators, flip cards, or data visualizations

Chatbots are capable of writing code in response to a question or prompt.

A prompt like “Build me a calculator with React” results in the following:

If you want to run this code, ChatGPT requires you to copy and paste the code into your IDE and run it on your own computer.

Claude goes one step beyond ChatGPT’s abilities with their Artifact system. Artifacts allow you to run the code within Claude’s interface and deploy to a shareable link. An unfortunate limitation is that you can’t make any direct edits to the code, so you’re entirely reliant on using prompts to make code changes.

(Perplexity is another tool you’ve probably heard of, which feels similar to chatbots but is more focused on search and not as useful for creating apps. It can write basic code because it’s built on top of other AI models like ChatGPT and Claude, but I wouldn’t recommend it for this use case.)

Remember that chatbots can write code for any part of our stack (client, server, database) but can’t host your code (deploy) for us. They also can’t create complex prototypes with multiple pages, and it’s difficult to change the code directly. As a result, these tools are best used for very simple one-time prototypes—which sometimes is enough to get the job done. Think of chatbots when you’re looking to create a very simple landing page, individual inputs like a date picker, or small apps like a to-do list.

Cloud development environments (Replit, Bolt, v0, Lovable)

Best for: Prototypes with more than one feature, specific design requirements, or many pages

Cloud development environments are one big step up from chatbots. These tools handle all the tasks required to turn your ideas into an actual working product. They can help you build end-to-end features, handle the backend infrastructure necessary to run your prototype, allow multi-file edits with agentic workflows, and take on more complex tasks across your codebase such as updating your database schema.

One of the key differentiators among the various cloud development environments is hosting. Every software product is built with three parts: a client, a server, and a database. The client is what the user interacts with (often written in JavaScript), the server processes requests from the client to retrieve data or integrate with other products (often written in Node.js, Python, or Java), and the database is your permanent storage of data. Making prototypes that have real features requires you to host both your client and server code, and may require a database to power the app.

One of the most popular tools today, v0, is capable of writing and hosting both client and server code. By default it uses specific frameworks called Next.js and Shadcn UI to do so (both were created by Vercel, the same company that owns v0). v0 can deploy your code and run backend servers—plus, one of its strongest features is that it has great styling as a default. Here’s a basic CRM I built in v0 with the prompt “Build me a basic CRM.”

Bolt is very similar to v0 in that it can also generate and deploy both client and server code. But a key difference is where the server runs. With v0, you deploy to real cloud hosting infrastructure, whereas Bolt runs the server code directly in the user’s browser. This means Bolt cannot natively support prototypes that need user identity like logins or accounts, multi-user interactions such as chat or collaborative workspaces, secure data operations like payment processing, or persistent data storage between sessions, because an isolated copy of the server is created on each user’s device. You can make up for this by integrating with external products like Supabase that offer servers and databases.

Here’s a basic CRM I built with Bolt, using the prompt “Build me a basic CRM.”

Another popular tool is Replit. Replit allows you to build full-stack applications, including a client, server, and database. It can build web apps using both JavaScript and Python frameworks and particularly excels at building internal admin tools (e.g. file conversion, job applicant tracking) and data-driven applications (e.g. image resizing, multi-page dashboards) with simple UIs.

I use Replit whenever I need a fully functional back end or I want to use Python code. I’ve used it to build an MP4-to-GIF converter and a Substack image resizer—both tools I use weekly.

Here’s a basic CRM I built with Replit, using the prompt “Build me a basic CRM.”

Finally, we have Lovable. It’s the newest of the bunch. Lovable is most similar to v0 and Bolt—it excels at generating websites, and uses JavaScript frameworks like React and Next.js. Its differentiation comes from its integrations with other popular tools. Lovable can connect to a GitHub repository, automatically add authentication and databases with Supabase, and help you connect to AI providers like Anthropic and OpenAI. All of these features make it one of the best AI coding tools for building products you actually want to use in production.

One major drawback of Lovable is the lack of a code editor. To edit code, you have to ask the agent with prompting. This can make it difficult to debug issues directly in Lovable. I often find myself starting a new feature here but moving over to Cursor to resolve problems.

Here’s a basic CRM I built with Lovable, using the prompt “Build me a basic CRM.”

To recap:

• Choose v0 for beautiful designs by default

• Choose Bolt for quick prototypes with flexible designs

• Choose Replit for internal tools or products that store or transform data

• Choose Lovable for building production apps that benefit from integrations with your current tools

Regardless of your choice, cloud development environments all support building more complex applications than chatbots, with the ability to deploy to the cloud and easily share updated iterations over time.

Local developer assistants (GitHub Copilot, Cursor, Windsurf, Zed)

Best for: People who know how to code and are working on serious applications they want to ship to production

The final type of AI development tool is local developer assistants. These products are targeted toward people who know how to write code. Tools like Cursor and GitHub Copilot can take prompts in a similar fashion to Claude but can then generate and apply changes within your own codebase and development environment (IDE). These tools do more than autocomplete—they can now write most of your code just using prompts.

For example, I built this presentation app (with live Q&A and polls!) in about 10 days using Lovable and Cursor. I started the app in Lovable to build basic features quickly, synced my code to GitHub to allow editing in other tools, and made final changes and fixed bugs with Cursor. This application uses authentication, databases, real-time updates, and more.

Ten days may sound like a lot, but most of that time was spent resolving bugs and troubleshooting issues—something Cursor excels at compared with other tools, and something I get into a couple of sections down.

GitHub Copilot is more popular in enterprise environments, as it comes from a trusted vendor, Microsoft. It supports multi-file changes from prompts, code explanations, and more. I’ve found it works best when given very specific direction and does not perform as well as Cursor at more general instructions. For example, when I asked for a new feature without providing context, Copilot re-created components of my app that already existed, whereas Cursor modified my existing files directly.

Two tools I haven’t spent as much time with but people are excited about are Windsurf and Zed. Windsurf is another IDE that can suggest multi-line changes to files and suggest commands such as moving files on your behalf. It excels at working on larger, more complex codebases. Zed is a highly performant editor built with a variety of productivity features such as prompt libraries, slash commands, and keyboard shortcuts for common actions like applying AI-generated code.

Building your prototype

Now that we’re familiar with the basic tools, let’s build some prototypes. The two most common prototyping use cases for product managers are:

1. Converting an existing design to a functional prototype

2. Building an idea into a prototype from scratch

Converting a design to a functional prototype

Let’s turn the following design for Airbnb’s home page into a working prototype. Say you want to use this prototype to explore a new feature, such as a price filter.

Here’s our initial design:

I chose Bolt for this task, as it’s better at building off a pre-existing design and we don’t need the backend database provided by Replit. Here’s a prompt I used (which you can copy and paste). Make sure to include a screenshot of the design!

“Build a prototype to match this design. Match it exactly.”

Here’s how this experience looks in Bolt:

Next, we’ll add our new price filter feature. Notice in the prompt below how I describe every feature in detail. One pro tip with prompting these tools is to be hyperspecific when describing changes for your subsequent prompts, as it helps the AI pinpoint what should change.

“Implement an inline price filter as a component of the search bar. It should appear next to ‘Add guests’ in its own section.

Selecting the input should pop up a price filter with minimum and maximum values. The background of the pop-up should be white and should cover elements beneath it.”

This is a great starting point. Let’s extend this feature with a slider for the minimum price.

“Can you add a price slider? It should have a blue line and a black node. Sliding the node should modify the minimum price.”

We now have a functional initial prototype of your product idea within 10 minutes! Without any coding skill. Incredible. We could even continue to improve this prototype filter (e.g. showing listings updated in real time as you adjust the price). Check out the prototype here.

Building a prototype from scratch

If you’re like me, your design skills are probably not good enough to create the initial design we used in the prior example. Luckily, you can build a prototype using existing patterns and components from free and publicly available design systems like Tailwind or Shadcn UI.

Let’s build a quick CRM with Bolt, then add a new feature to it. Let’s say we’re considering adding a feature that automates email outreach directly from our CRM and want to gather customer feedback about this feature before building it, by showing potential users a prototype of what it might look like.

We can quickly put together a v1 with the following prompts:

“Create a comprehensive customer relationship management (CRM) system.”

Again, let’s take a moment to pause here. We just created a working prototype of a CRM in less than five minutes—something that would have previously taken weeks of an engineer’s time. Unbelievable.

Let’s keep going and see how we add a new feature.

“Please implement a mock AI email writer. This should be accessible from the left nav.”

Again, less than five minutes to be able to play with a new feature idea. Just think about how many ideas you can explore and how quickly you can bring them to market. AI prototyping tools have been available for less than six months and have already absolutely changed the speed at which teams can ship.

Now we can take our prototype and get customers’ direct feedback without wasting time developing an initial version. This approach can speed up your discovery process by getting interactive examples in your customers’ hands as early as possible.

Each AI tool will have very different outcomes based on their default settings and how specific you are. Here’s the exact same example with the same prompts using v0:

Common use cases and prompt templates

Here are a few good templates you can use to get started with each of the tools I’ve highlighted:

Task #1: Build a prototype from an existing Figma design

Prompt:

Build a prototype to match this design. Match it exactly. Use Tailwindcss.

Match styles, fonts, spacing, and colors.

[Include a single screenshot from Figma]

Tool: Bolt

Example: Deployment manager