This is the Ebbinghaus Forgetting Curve in action. Your brain isn’t a tape recorder; it’s a storyteller that fills in gaps with assumptions. If you’ve ever spent a week building a feature only to realize the VP never actually asked for it, you’ve experienced the “Shared False Memory” trap.

The Group Hallucination: Why Your Roadmap is at Risk

It’s one thing to forget a meeting; it’s another to collectively “invent” one. In groups, this phenomenon turns dangerous. Research shows that teams often walk out of a room with shared false memories, convinced the CEO issued a directive that she never actually said.

You implement a non-existent strategy, and six months later, no one even questions it because everyone “remembers” it clearly. I’ve seen a client team waste three entire weeks building a feature their VP supposedly “asked for”. The transcript later proved she never mentioned it, yet four senior people would have sworn on their careers that she did.

The hard truth?

Your memory is just a piece of fan fiction you wrote about the meeting after it ended.

The Failed Fixes of the Past

We’ve all tried the standard “productivity” hacks, and we know why they fail:

• Manual Note-taking: You split your focus. While you’re typing the last sentence, you’re missing the next one. You capture words, but you aren’t actually present to lead the conversation. And let’s be honest: you’ll never look at those notes again.

• The “Audio Graveyard”: You hit record, creating 47 minutes of audio that will sit in a folder until the end of time. We both know you aren’t going to replay it.

• The Bot Invasion: You invite an AI notetaker, and suddenly the vibe shifts. People talk differently when a bot is watching. You end up with a generic “nothing-summary” that tells you the team “discussed priorities”—which helps exactly no one.

You eventually give up and Slack a colleague: “Hey, what did we actually decide?”.

I’ll say it plainly: I hate AI notetakers.

You know the ones. You join a Google Meet or Zoom, and instead of a focused group of humans, you see a dozen “Assistants” and “Recorders” staring back at you. It’s a nightmare straight out of a Black Mirror episode.

When a bot joins, the room changes. People stop speaking freely. They filter their thoughts because they know a “gray circle” is transcribing their every word for a generic summary you probably won’t even read. We’ve reached a point where we should almost shame the use of these intrusive bots—they destroy the very “realness” of the conversation they’re trying to capture.

For a long time, I thought note-taking simply wasn’t a solved problem. I went back to scribbling poor, disjointed notes just to avoid the “Bot Invasion.”

Then, I found Granola

The “Invisible” Advantage: Why Granola is Different

Unlike typical AI notetakers, Granola is a local app. It doesn’t join your call as a participant. It simply listens to your device audio, transcribing in the background while you stay 100% present in the conversation.

• The "Ghost" Mode: No bot joins the call. Granola runs locally, capturing audio directly from your device. Because there is no "AI Notetaker" circle visible to others, the conversation remains natural and authentic. You get the real story, not the filtered version.

• Privacy First: Your data remains yours. Granola doesn’t store your audio—only the transcripts, which you control completely. This allows you to stop typing entirely and stay fully engaged in the meeting.

• Fully Focused: You don’t have to type a single word during the call. You can finally look your customers in the eye (digitally) and catch the nuance you’d usually miss while frantically taking notes.

• The Power of "Recipes": Beyond being invisible, Granola allows you to build custom prompts—what they call "Recipes"—to automate your most tedious tasks.

I know you’re already thinking about it: you want to copy my personal prompts. This is the #1 reason I’ve abandoned every other AI tool.

No More Copy-Paste Fatigue

The genius of Granola is its built-in AI chat. You don’t need to export text or juggle tabs with Claude or ChatGPT. You simply type “/” directly within your transcript to run a Recipe and get your output instantly.

<video>

Prompt 1: TLDR so-short-you-have-to-read it.

Default AI summaries are three paragraphs of nothing.

I need two bullets I’ll actually look at.

Link for direct access → https://recipes.granola.ai/r/7bc211bc-6278-4a78-a56b-d45630b72056 But if you want to recreate it, here’s the exact prompt I use:

You are summarizing a meeting for someone who is extremely busy and will only read something if it’s shockingly short. Your job is to distill the meeting provided into a single TLDR that is so brief, skipping it would feel ridiculous.

RULES:

• Maximum 2 bullet points total

• Each bullet must be under 12 words

• No fluff, greetings, or preamble

• No “In summary...” or “Key takeaways include...” — just the points

• Only include what actually matters: decisions made, actions required, or critical information

• If nothing important happened, say “Nothing worth your time.”

FORMAT:

• [Most important thing, the 80%]

• [Second most important thing, the 20%]

That’s it. Nothing else.

What you get back:

• Aman owns the pricing proposal by Friday.

• We’re not launching in Q1 anymore.

That’s it. The meeting in ten seconds.

Quick How-To for Granola Recipe.

How to create your own Granola Recipe:

1. Go to your Granola account.

2. Go to the bottom left “Recipe”.

3. Click on “Create a Recipe on the top right”.

And then it’s quite simple.

1. Type your name at the top (you will access it with / ).

2. The prompt part is the most important one. Copy mine or do yours.

3. Select the model. My favorite is claude-opus-4-5.

4. Make it for single meetings or multi-meetings.

PS: If you struggle to make your own Recipe, leave a comment.

I’ll help you prompt it.

Leave a comment

Prompt 2: Action items, nothing more.

Default action items are vague. “Follow up on discussion.”

Follow up on what? With who? By when?

I made a Recipe for that. My action items, nothing more.

The link for direct access → https://recipes.granola.ai/r/43347a99-e934-4972-8a88-cef1b9c7b27d

Or here’s the exact prompt of my Recipe:

You are reviewing a meeting to extract action items assigned to a specific person. Your job: Find and list of the action items assigned to the people the user will mention. It could be multiple people.

Rules:

- Output action items as bullet points only

- No introduction, greeting, summary, or closing remarks

- No headers or section titles

- If an action item has a deadline mentioned, include it in parentheses at the end

Example of an output format:

[name]

- Send the updated contract to legal (by Thursday)

- Book the venue for the offsite

- Review David’s proposal and share feedback

- Circle back with the client about pricing

Now I know exactly what I owe. Just what was actually said.

The “I never forget meeting” workflow.

1. Meeting starts.

2. Granola takes notes on the background.

3. Meeting ends.

4. Open Granola.

5. Type “/” → run the TLDR prompt → 2 bullets essence.

6. Type “/” → run the action items prompt → copy to Slack or Notion.

7. Done.

Two minutes. Every meeting.

No more “wait, what did we decide?”

You finally remember more than 30% after a day.

And you can still hate on AI notetakers bots. Shame them, really.

If you agree, say it louder in the comments.

We can all come up with great ideas for LLMs to enhance our products, but the devil is in the details:

• How do we get our AI to adopt a very specific writing style?

• How do we get our AI to use our latest product documentation?

• What’s the fastest way to test an AI concept and get it into users’ hands for feedback?

These aren’t just technical questions. They’re strategic product decisions.

Everyone can access powerful foundation models like GPT-4.5 or Claude Opus 4. So, what’s going to make your AI feature stand out?

Hint: It’s not just about having ‘AI’.

It’s about making that AI uniquely yours and genuinely useful.

The 3 Approaches to Optimizing AI

Base LLMs are like brilliant interns: incredibly capable, but they don’t know your company’s specific jargon, your proprietary data, or the nuanced style your customers expect.

Leaving them “off-the-shelf” often leads to:

• Hallucinations & Generic Answers

• Lack of Domain Expertise

• Inconsistent Brand Voice

You have three major ways to address these problems:

1. RAG (Retrieval Augmented Generation)

   Have a model do a search to enhance its results with information not in its training data set, then incorporate those findings into its answer

2. Fine-tuning

   Specializing a model based on information

3. Prompt engineering

   Better specifying what you’re looking for from the model

The question is: when should you use each, and why?

Today, we’re breaking down the definitive framework for choosing between prompt engineering, RAG, and fine-tuning.

Today’s Post

We’re going to give you the context, decision frameworks, and a practical step-by-step walkthrough to help you build the intuition. Once you build each on your own, you’ll have that AI engineer level knowledge to speak with them confidently:

1. Mistakes

2. Pros and Cons

3. Decision Framework

Think of this post more like a course-lesson where you have to follow along than an article.

TL;DR: Start with prompt engineering (hours/days), escalate to RAG when you need real-time data ($70-1000/month), and only use fine-tuning when you need deep specialization (months + 6x inference costs).

1. The 3 Major Mistakes We See

Let’s start by understanding what not to do, then build up to the right approach.

The Strategic Imperative

The truth? There’s a strategic choice between Prompt Engineering, RAG, and Fine-Tuning.

Choosing RIGHT? That’s your path to AI-driven product growth.

Too often, teams jump to:

1. Over-engineering: “Let’s fine-tune a massive model!”

   (Costly, time-consuming, maybe overkill)

2. Under-powering: “We’ll just tweak the prompt a bit more.”

   (Hits a ceiling, can’t handle complex needs or new data)

3. The “Black Box” Hope: “We’ll plug in an API and hope for the best.”

   (Lacks control, struggles with proprietary knowledge)

The truth is, there’s a strategic choice to be made between Prompt Engineering, Retrieval Augmented Generation (RAG), and Fine-Tuning.

Choosing wrong means wasted cycles, blown budgets, and features that underwhelm.

Choosing right? That’s your path to AI-driven product growth.

This article is your map.

2. Pros and Cons: RAG vs. Fine-tuning vs. Prompt Engineering

Let’s start by understanding each of the three techniques at a deep conceptual level and discuss their pros and cons.

Prompt Engineering

As upcoming podcast guest Hamel Husain says:

Prompt engineering is just prompting these days.

We all have to prompt, but when it comes to building the right prompt for your product feature, prompt engineering is critical.

It goes beyond simple clarification. It’s about transforming the model’s output with additional training or data retrieval. It’s about better activating a model’s existing capabilities.

Pros:

• You don’t need to change backend infrastructure

• You get to see immediate responses and results to what you do - no new training data or data processing required

Cons:

• Trial and error - it’s as much an art as a science

• You’re limited to the model’s existing knowledge, unable to add new or proprietary data

Fine-tuning

Fine-tuning takes an existing foundation model and gives it specialized ‘graduate-level’ training on a focused dataset relevant to your specific needs.

You’re subtly adjusting the model’s internal ‘weights’ (its understanding of relationships in data) to make it an expert in a particular domain, style, or task.

This typically involves providing hundreds or thousands of high-quality input-output examples.

Pros:

• Great when you need deep domain expertise or consistent tone/style

• Faster at inference time than RAG because it doesn’t need to search through external data and don’t need to maintain a separate vector database

Cons:

• Issues with the training complexity - need 1000s of examples

• There are significant computational and maintenance costs involved

• You risk “catastrophic forgetting,” where the model loses some general capabilities as it becomes more specialized

RAG

Retrieval Augmented Generation is like giving your LLM real-time access to a specific, curated library of information – your product docs, a knowledge base, recent news, etc.

When a user asks a question, the RAG system first retrieves relevant snippets from this external library and then feeds that context to the LLM along with the original query.

The LLM then uses this fresh, targeted information to generate its answer.

Pros:

• Good for up-to-date information

• Good for adding domain-specific information

Cons:

• Performance impact - retrieval adds latency to each prompt (typically 100-500ms)

• Processing costs - eg for the vector database

Ready to add more context to this? We highly recommend staying around for section 4 - where we walk you step-by-step through all 3 so you can have a great addition to your PM portfolio and learn the intuition behind this.

3. Decision Framework

Now comes the million-dollar question: how do you actually decide which one to use, and when?

This is our battle-tested framework after learning from case studies at Shopify, Google, Apollo, and OpenAI:

The Million-Dollar Mistake Most Teams Make

Here’s the uncomfortable truth, the one that burns through budgets and timelines faster than a poorly optimized LLM: Teams reflexively jump to the most complex solution.

Fine-tuning sounds sophisticated, like you’re truly ‘building AI.’

RAG systems feel robust, like you’re tackling data head-on.

So, engineers get excited, data scientists propose intricate architectures, and suddenly, you’re six months deep into a project that could have delivered value in six weeks.

Worst of all? They try to do everything at once.

At Shopify, we’ve seen this play out firsthand with ‘Auto Write,’ their AI content generation feature.

The initial internal buzz was all about fine-tuning GPT-3. Data science was already sketching out a sophisticated RAG system. Everyone had a strong opinion, usually leaning towards the most technically challenging path.

What actually worked? What shipped and delivered value?

They started with disciplined prompt engineering. Simple, direct, and focused.

The result?

They shipped a high-accuracy feature in 10 weeks.

The fancy, more complex approaches like RAG and fine-tuning came later – much later – only after they had validated the core user problem and proven the initial value with the simplest effective method.

The lesson is: don’t let the allure of complexity derail your path to impact.

Our Practical Decision Framework: Your Step-by-Step Guide

How do you avoid the ‘Million-Dollar Mistake’ and make the right call? Here’s the battle-tested decision framework Miqdad and I use:

Let’s walk through the logic.

Step 1: Nail Your Use Case

Before you write a single line of code or craft a prompt, ask yourself: Is the user problem you’re aiming to solve with AI crystal clear? Do you know precisely who this is for and what a “win” looks like from their perspective?

Vague goals breed vague AI – features that meander, require endless tweaks, and ultimately, satisfy no one.

“AI for sales” isn’t a strategy; “AI to draft personalized follow-up emails for SMB sales reps based on CRM interaction history” is getting warmer.

If your use case is fuzzy, stop. Seriously. Hammer out the problem definition, user stories, and key success metrics.

This isn’t just a nice-to-have; it’s the non-negotiable foundation. Only once your target is sharp should you proceed.

Step 2: The Prompt Engineering Gauntlet

With a clear use case, your first port of call is always prompt engineering. Can you get to roughly 80% of your desired outcome with well-crafted prompts alone?

This is where you test your MVP or prototype ideas. It’s your fastest, cheapest way to see what the base model is capable of.

Don’t underestimate what good prompting can achieve.

But also, don’t fall into the trap of spending months trying to coax magic out of prompts for a task that fundamentally needs more.

Give yourself a tight timebox – say, one to two weeks max of dedicated effort – to explore this. If you hit that 80% mark, fantastic! Ship it. Get it in front of users, gather feedback, and iterate.

Step 3: The Data Question – Does Your AI Need a Live Feed?

If prompts alone aren’t cutting it, the next question is about data.

Does your AI feature absolutely depend on information the base model couldn’t possibly know? Think: data that changes by the minute, your company’s latest internal product documentation, or user-specific context.

Trying to “teach” an LLM constantly shifting facts through fine-tuning is like trying to fill a leaky bucket – slow and inefficient.

And while prompts can handle small bits of context, they can’t effectively inject vast, ever-changing datasets.

If your AI needs to be current and deeply aware of your specific, evolving world, then RAG (Retrieval Augmented Generation) is your answer.

It gives your LLM a live connection to the knowledge it needs, precisely when it needs it. If external data isn’t the core issue, your challenge likely lies in shaping the model’s inherent behavior or style.

Step 4: The Fine-Tuning Fuel Check – Got Quality Data?

So, prompts weren’t enough, and external data access (RAG) isn’t the primary bottleneck.

You’re now likely looking to instill a very specific style, tone, or a complex, nuanced understanding that goes beyond simple information retrieval.

This is often the case for achieving high volume and consistency in outputs, or for tackling enterprise-level tasks demanding sophisticated pattern recognition.

Welcome to the doorstep of fine-tuning.

But before you step through, check your fuel tank. Do you have at least a thousand (and often many more) high-quality training examples?

These aren’t just any examples; they need to be carefully curated, accurate input-output pairs that perfectly demonstrate the desired behavior you want the model to learn.

Fine-tuning with a handful of examples, or with messy, low-quality data, is a recipe for disaster.

Garbage in, garbage out is brutally amplified here. If you don’t have this data, your immediate focus should be on collecting and curating it, even while you continue to iterate with prompts or RAG.

Step 5: The ROI Litmus Test for Fine-Tuning

You’ve got the data. The need seems clear. But now, the crucial business question: Is the return on investment for fine-tuning undeniably compelling?

Fine-tuning “because we can” or because it sounds impressive on a slide deck is how innovation budgets die a quiet death. The bar should be high.

If the value proposition isn’t screamingly obvious, it’s probably not the right time. Stick with your refined prompt and RAG setup. Re-evaluate later when the case is stronger.

But if the data, the need, and the ROI all align? Then you’ve earned the right to strategically deploy this powerful, resource-intensive technique.

The Core Philosophy: Start Simple, Add Complexity Deliberately

The takeaway here isn’t subtle: Always start with the simplest approach that could work.

Prompt engineering is your first, best friend. Only escalate to RAG or fine-tuning when the problem genuinely demands it, and you can clearly articulate the business case for the added complexity and cost.

Stay tuned for Part 3…

We have mastered instant home deliveries. We have solved door-to-door travel. We have built entire industries around helping people earn and save money.

But what happens after we save that time?

Often, a second wave of companies—social media, endless streaming, doom-scrolling apps—sweeps in to capture that “saved” time.

We saved minutes on groceries only to lose hours to distraction.

Even education companies, which promise better careers and higher salaries, often result in corporate friends who are burnt out and unhappy.

The actual problem isn’t efficiency. It’s happiness.

And frankly, most technology today doesn’t know how to solve that.

In that post, I suggested three starting points to fix this:

1. Mental Therapy (Access to care)

2. Humanizing Business (Empathy in work)

3. Tech Empowerment (Tools that heal rather than distract)

Today, I want to talk about that third point: Tech Empowerment.

Subscribe now

Building the Solution

As we step into the New Year, I didn’t want to just make resolutions. I wanted to build a tool that helps keep them.

I realized that while I cannot replace a human therapist, I can use the latest advancements in AI agents to make support more accessible, immediate, and structured.

This week, I built a Multi-Agent Mental Wellbeing System.

Unlike a standard chatbot that gives you generic advice, this system mimics a supportive care team. It is designed to help you deal with the chaos of life—whether it’s job stress, relationship anxiety, or just that overwhelming feeling that comes with a new year.

How It Works (and How It Helps You)

The tool isn’t just one AI; it is a “swarm” of three specialized agents working together to support you:

1. The Listener (Assessment Agent): We often just need to be heard. This agent doesn’t try to “fix” you immediately. It listens. It analyzes your emotional state, sleep patterns, and stress levels to understand exactly where you are standing right now.

2. The Helper (Action Agent): Once you’ve been heard, you need clarity. This agent cuts through the noise. It provides immediate, concrete steps—whether that’s a breathing exercise for right now or a resource to call tomorrow. It turns anxiety into action.

3. The Partner (Follow-up Agent): This is the piece most tech misses. We don’t just need a quick fix; we need a lifestyle change. This agent builds a long-term strategy for you, focusing on habits and sustainable routines.

A Tool for the New Year

The New Year is often a source of pressure. We set massive goals and feel like failures by February.

I want this agent to be a tool for resilience, not just productivity. It is an example of technology that doesn’t demand your attention but supports your intention.

If you are feeling the weight of the world, or just the weight of a hectic job, give it a try. Let’s use technology to reclaim not just our time, but our peace of mind.

Here’s to a happier, healthier, and more empowered 2026.

Adityeah

Subscribe now

P.S. For the Builders

For those interested in the architecture, this system uses AG2 (formerly AutoGen) to orchestrate the agents. By separating “Empathy” (Assessment) from “Logic” (Action), we drastically reduce hallucinations and improve the quality of care.

You can check out the open-source code here:

GitHib Link

In one world, the tech has never been better. Chips are faster, models are stronger, and building AI features is easier than ever.

In the other world, the features we ship often fall short. Many AI tools still feel like junior interns—helpful at times, but needing guidance and making mistakes you don’t expect.

And most of the time, the issue isn’t the model itself. It’s the context we give it.

That’s why context engineering matters. It may sound technical, but it’s quickly becoming one of the most important skills for anyone building AI products.

Today’s Post

1. Why Does Context Engineering Matter?

2. The PM’s Role in Context Engineering

3. The 6 Layers of Context to Include

4. Common Mistakes

5. How to Engineer Context Step-by-Step

6. How to Spec out Features Appropriately

7. Checklists, templates + prompts you can steal

1. What is Context Engineering, and Why Does it Matter?

What Is Context Engineering?

Context engineering is the practice of giving an AI model the exact information it needs at the exact moment it needs it. Think of prompt engineering as writing instructions, while context engineering builds the world those instructions live in.

Humans naturally infer context—we remember who we’re talking to, what just happened, what matters, and what rules apply. LLMs don’t. The models don’t magically know:

• who the user is

• what happened a few seconds ago

• which document matters

• what the system already knows

• what rules or policies apply

• what data they’re allowed to access

• what was said in previous sessions

• whether the user is new or experienced

• what constraints must be followed

• which entities exist in the workspace

• how different pieces fit together
  

None of this comes for free. We have to supply it.

That is the job of context engineering—and it’s the difference between an AI that feels clueless and one that feels intelligent.

Why Context Engineering Matters?

Most people obsess over models and prompts.

“Should we use GPT-5.1 or Gemini 3?”
“What’s the perfect prompt?”

Those questions matter—but only after you’ve done the hard work of context.

Because if your system:

• doesn’t know which file the user is working on

• doesn’t see the user’s preferences

• isn’t aware of entities and relationships in the workspace

• can’t recognize the user’s role

• retrieves random, irrelevant documents

• or misses key logs and events

…then it doesn’t really matter which model you plug in. The output will still feel off.

Context is what turns “a smart model” into “a useful product.”

Example 1: AI Email Assistants

When I was working on an AI email writer, I learned this the hard way.

What the model saw changed everything:

• Only the last message → the reply was generic

• Full email thread → the reply became coherent

• Thread + CRM notes → the reply became personalized

• Thread + CRM + tone-of-voice guidelines → the reply felt on-brand

• Thread + CRM + tone + relationship context (e.g., “key customer,” “renewal risk”) → the reply became shippable

Same model. Very similar prompts. Completely different quality.

That’s context engineering at work.

Example 2: AI Coding Assistants (Cursor)

Take coding assistants.

Ask yourself: why hasn’t Cursor been steamrolled by every new “better” model release, and instead grown into a billion-dollar+ business?

The key is not just which model they use. It’s how they prepare context.

Roughly speaking, tools like Cursor:

1. Index your codebase:

   • Split the code into meaningful chunks (often using the AST).

   • Compute embeddings for every chunk.

2. Understand your question:

   • Turn your query into embeddings.

   • Search a vector database for the most relevant files, functions, and lines.

3. Build a rich context window:

   • Pull in only the relevant snippets.

   • Feed those into the model along with your request.

Because the context is so strong, Cursor can let you switch between OpenAI, Anthropic, Gemini, xAI, etc. The model is almost a plug-in.

The moat is the context layer: the indexing, retrieval, and wiring of code + queries into something the model can reason over.

That’s also why you see huge companies trying to buy or replicate these players instead of just “turning on a better model.” Model capability alone is not enough. Context is what makes the product defensible.

Now that we’ve talked about why context engineering matters and how it shows up in real products, the rest of this guide will focus on the “how”:

• frameworks and canvases you can use as an AI PM

• real-world checklists

• templates and prompts you can copy straight into your own products

The goal: help you treat context engineering as a first-class product skill, not an afterthought.

2. The PM’s Role in Context Engineering

Many teams assume context engineering is something only engineers handle.

It isn’t.

Context engineering sits where product thinking, user insight, and system design meet. Engineers can build the plumbing, but they can’t decide which signals matter, how they shape the experience, or what “good” looks like for the user.

That judgment lives with the PM.

What PMs Own in Context Engineering?

As a PM, there are three layers you’re directly responsible for—layers engineers cannot decide on their own.

1. Defining what “intelligence” means for the feature

Before anything is built, you must spell out what success looks like. That includes:

• What the AI should know about the user

• What domain knowledge is essential vs. optional

• Which user actions should update the context

• How much personalization creates value without crossing the line

These aren’t technical calls—they’re product calls.

2. Translating user value into context requirements

Users express fuzzy needs. PMs convert those into concrete context specifications.

• “Users want better suggestions” →
  system needs access to past dismissals, workspace state, and team preferences

• “Make it feel personalized” →
  capture writing style, common corrections, and role-specific patterns

Without this translation, engineering builds general-purpose systems that function but don’t feel smart.

3. Designing how the system behaves when context is missing

Context will fail: documents won’t load, signals will be stale, permissions will restrict data.

You decide the fallback behavior:

• Should the feature block?

• Should it return a partial answer?

• Should it ask clarifying questions?

• Should it switch to a simpler mode?

This is core product judgment. It defines reliability, trust, and user perception.

What Engineers Own?

Engineers handle the execution:

• retrieval architecture

• vector databases

• embeddings and chunking

• indexing pipelines

• integrations

• latency, reliability, and scaling

But all of this depends on the PM defining what to retrieve and why it matters.

The Division of Labor

A simple way to think about it:

• PMs define the context pyramid — what information belongs at each layer

• Engineers build the infrastructure — how to store, fetch, and update it

• PMs design the orchestration logic — what the model should see and when

• Engineers implement the orchestration engine — the system that delivers it in real time

When PMs skip their part, teams end up with technically impressive systems that feel oddly unintelligent—because nobody defined what intelligence actually required.

3. The 6 Layers of Context to Include

If you look closely at every world-class AI product, you’ll find their intelligence doesn’t come from clever prompting or bigger models. It comes from a carefully engineered hierarchy of context.

LAYER 1 — INTENT CONTEXT

Starting at the foundation…

Deciphering the user’s actual goal, not just their literal words.

Nearly every AI disaster—hallucinations, logic errors, bad suggestions, irrelevant answers—stems from a single failure point: the system misread the user’s objective.

Humans are messy communicators.

They say one thing but want another. They highlight a paragraph without explaining the error. They click, pause, rewrite, or use shorthand that relies on assumptions the model cannot see.

This is the role of Intent Context: to give the AI interpretive intuition—the ability to reconstruct the user’s true aim, even when the prompt is vague, incomplete, or emotional.

The “Triple-I” Decryption Model

To turn intent from a concept into code, every input must pass through a three-stage filter:

1. Interpret: Convert the raw text into a clear, structured objective.

2. Infer: Analyze recent actions (clicks, hovers, deletions, retries) to find the subtext.

3. Identify Gaps: Spot missing data points (files, code snippets, metrics) that must be fetched before answering.

This converts messy input into machine-ready intent, clearing the fog so the next five layers can work.

LAYER 2 — USER CONTEXT

A live profile of the human: their habits, quirks, role, and mental model.

Even a technically perfect answer fails if it feels robotic.

A principal architect wants the raw diff; a junior developer needs the tutorial. A CEO wants the bottom line; a researcher wants the citations.

User Context ensures the AI acts not as a generic chatbot, but as a personalized extension of the operator—matching their rhythm, tone, expertise, and history.

The 5-P Personalization Matrix

To make this practical, track these five variables for every user:

1. Preferences: Voice (terse vs. warm), verbosity, style.

2. Patterns: How they usually edit, format, or correct outputs.

3. Proficiency: Expert vs. Novice (dynamic complexity adjustment).

4. Pacing: The speed at which they consume information.

5. Purpose: Their specific job-to-be-done and workflow goals.

This allows the AI to speak in the user’s language, rather than the LLM’s default setting.

LAYER 3 — DOMAIN CONTEXT

Without grounding, an AI isn’t a product; it is a hallucination engine with a nice UI.

Domain Context transforms your system from a creative toy into a subject-matter expert. It enables the AI to cite real objects, respect dependencies, use correct terminology, and navigate your company’s reality.

This is the “Source of Truth”: your entities, metadata, dependency trees, documentation, codebases, business logic, and institutional history. But you cannot just dump raw text. It must be structured.

Every domain requires five structural anchors:

1. Entities: The nouns of your world: tickets, users, dashboards, repos.

2. Attributes: The descriptors: dates, owners, tags, status.

3. Relationships: The web: “blocks,” “enables,” “owns,” “is part of.”

4. Definitions: The logic: canonical formulas and business rules.

5. Lineage: The history: who changed what, and when.

LAYER 4 — RULE CONTEXT

The laws, limits, and guardrails that dictate what the AI allows.

Intelligence without constraints is dangerous. Rule Context acts as the judiciary of your system: deciding what is permitted, what is required, and what is strictly banned.

This covers everything from output schemas and permissions to safety compliance and formatting mandates. These cannot be polite suggestions in a prompt—they must be hard boundaries.

The Dual-Barrier Framework

Enforce rule context using two types of walls:

1. The Soft Wall: Advisory guidelines (brand voice, style, tone).

2. The Hard Wall: Mandatory requirements (JSON schemas, security, permissions).

Soft walls guide the vibe. Hard walls guarantee the function.

Together, they turn the AI from a probabilistic guesser into a deterministic operator.

LAYER 5 — ENVIRONMENT CONTEXT

The immediate reality surrounding the user right now.

Most work depends on the “now,” not on static knowledge.

A coding bot needs to see the open tab. A writing assistant needs the selected sentence. A support bot needs the error log from one minute ago. A planning bot needs the deadline expiring today.

Environment Context gives the AI situational awareness—knowing the active file, the cursor position, the timestamps, and the current system state.

The N.O.W. Awareness Model

Capture the real-time signal via three channels:

1. Nearby Activity: The user’s immediate focus (selection, cursor, scroll).

2. Operational State: System health, logs, errors, device info.

3. Window of Time: Urgency, deadlines, and recency.

This stops the AI from acting generically and helps it act contextually—like a coworker looking over your shoulder.

LAYER 6 — EXPOSITION CONTEXT

The final, polished, noise-free signal the model actually receives.

This is the top of the stack—the point where the previous five layers merge into a clean, ranked, coherent payload.

Exposition Context is where potential becomes performance.

It is the difference between handing the model a library and handing it the exact page it needs.

The Context Distillation Pipeline

Every prompt payload must survive a five-step purification process:

1. Collect: Aggregate signals from Intent, User, Domain, Rule, and Environment.

2. Compress: Cut the fluff, remove redundancy, resolve conflicts.

3. Construct: Organize the data into clear, labeled sections.

4. Constrain: Apply safety checks and formatting rules.

5. Check: Verify the payload is consistent and ready for reasoning.

Unifying the Stack

When these six layers align, the AI wakes up. Most products skip steps or build them poorly. The winners treat every layer as critical infrastructure, not optional features.

4. The Divide: Engineering Context vs. Dumping Data

If you analyze high-performing AI agents, you will find they share a distinct architectural blueprint. Conversely, systems that hallucinate, drift, or confuse users almost always suffer from the same set of structural flaws.

The difference isn’t the model; it’s the discipline of the context pipeline.

What Great AI Products Do Right

World-class AI products treat context as a strict engineering discipline, adhering to these principles:

1. Structure Over Chaos: They never feed the model raw blobs of text. They invest heavily in schemas, metadata fields, and entity graphs because they know that unstructured inputs inevitably yield unstructured behavior.

2. Surgical Curation: They do not “give the model everything.” They aggressively pre-filter, selecting only the specific signals required for the current reasoning step. They value signal-to-noise ratio over total volume.

3. Explicit Segmentation: The model receives context in clear, labeled packets (e.g., <UserHistory>, <SystemConstraints>, <ActiveFile>). They use the six layers to feed information in a predictable hierarchy, reducing ambiguity.

4. Deterministic Guardrails: They do not rely on the LLM to police itself. Hard constraints are enforced by external validators and code logic, not by polite requests buried in the prompt.

5. Dynamic Knowledge Graphs: They treat domain knowledge as a living network. Every new document or interaction updates the relationships in the graph, ensuring the context evolves alongside the business.

6. Multi-Step Reasoning: They rarely rely on a single “God Prompt.” Instead, they use chains: planning steps, summarization calls, and iterative refinement loops to build the answer piece by piece.

The Architecture of Failure

Broken systems are equally predictable. Across hundreds of failed implementations, these patterns appear repeatedly:

1. The “Kitchen Sink” Strategy: Stuffing the entire context window with every available document. This leads to token overload, model confusion, and “lost in the middle” retrieval errors.

2. Naive Semantic Search: Relying solely on vector search (RAG) without structural filtering. This retrieves conceptually similar but factually irrelevant chunks that derail the model’s logic.

3. The “Prompt-as-Code” Fallacy: Assuming that writing “Do not do X” in a prompt is a security measure. Prompts are probabilistic suggestions; without hard code boundaries, the model will eventually ignore them.

4. Source Amnesia (No Provenance): Feeding data to the model without tracking where it came from. When the AI cannot cite its sources or distinguish between an old memo and a new policy, reasoning becomes impossible.

5. The “Magic Box” Assumption: Expecting the LLM to infer the domain structure on its own. No matter how smart the model, it cannot guess your internal business logic unless you explicitly define the schema.

The Bottom Line

Great context engineering is about discipline. It requires rejecting the shortcut of “just dumping data” into the prompt and instead building the rigid infrastructure—schemas, layers, and filters—that true intelligence requires.

5. How to Engineer Context Step-by-Step The C.E.O. Framework

The previous section covered what context is. This section covers how to build the invisible machinery that manages it.

Truly intelligent AI doesn’t just “react” to a prompt. It runs a continuous background process that gathers raw signals, structures them into meaning, and carefully curates what the model sees.

This is the C.E.O. Framework: Capture, Enrich, and Orchestrate.

PHASE 1: CAPTURE

How the System Absorbs the World

“Capture” is the foundational act of noticing. It is the sensor array that collects the raw ingredients of intelligence from the user, the application state, and the environment.

In high-quality AI products, this happens passively. The system does not wait for the user to explain their situation; it silently observes the micro-signals that define intent. It notices:

• The document currently open.

• The specific text highlighted or the field just edited.

• The filters applied to a dashboard.

• The segment drilled down into during analysis.

• The last output the user accepted (or rejected).

• Looming deadlines on the timeline.

The Three Streams of Signal To build a complete picture, a robust capture layer aggregates three distinct families of inputs:

1. Explicit Signals: What the user intentionally provides (prompts, specific selections, uploaded files).

2. Implicit Signals: What the user subconsciously reveals (scroll depth, cursor hover time, recent click paths).

3. System Signals: The environmental reality (timestamps, object metadata, data freshness, error logs).

The Goal: To transform a stream of fragmented micro-data into a coherent answer to the question: What is actually happening right now?

PHASE 2: ENRICH

How Raw Signals Become Structured Meaning

Raw data is noisy; enriched data is actionable.

Enrichment is the process of converting ambiguous, unstructured signals into a consistent schema the model can reason about. It turns the messy complexity of human behavior into a clean “context package.”

This involves extracting entities, normalizing fields, resolving ambiguity, and filtering noise. It is the difference between the system seeing “User clicked the button” and seeing “User approved the Q3 Financial Report.”

The Engineering Challenge: Reconciling Reality Enrichment is difficult because reality is inconsistent. The system must handle:

• Ambiguity: Two users using the same phrase to mean different things.

• Vague References: A user asking about “the latest update” (which requires checking timestamps against version history).

• State Drift: A metric changing in the background while the user was working on something else.

The Transformation Pipeline A sophisticated enrichment layer performs four critical operations:

1. Disambiguation: Clarifying references like “this issue” or “that customer.”

2. Stitching: Connecting related objects (e.g., linking a PR to the Incident Ticket it solves).

3. Inference: Reconstructing implied relationships (e.g., “The user selected a high-risk item, implying they are in a risk-mitigation workflow”).

4. Augmentation: Injecting domain knowledge (e.g., “This metric relies on Stream X, which is currently down”).

PHASE 3: ORCHESTRATE

How the System Decides What the Model Sees

Orchestration is judgment.

Just because you captured and enriched the data doesn’t mean the LLM should see all of it. Feeding a model too much information confuses it; feeding it too little hallucinates it.

Orchestration is the deliberate decision-making layer that determines exactly which artifacts are included, in what order, and in what format.

The Four Balancing Forces The orchestrator must balance four competing constraints simultaneously:

1. Relevance: Does this signal materially affect the answer, or is it noise?

2. Brevity: Can we fit this within the token window without losing nuance?

3. Precision: Is the structure clear enough for the model to parse?

4. Timing: Is this the right information for this specific step of the chain?

The Architecture of Control This is where context engineering becomes an art form. A strong orchestration layer separates concerns into distinct logic blocks:

• The Filter: Selects the most relevant artifacts from the domain graph.

• The Structurer: Organizes data into labeled sections (summaries vs. raw text).

• The Ruler: Enforces constraints and business logic.

• The Personalizer: Injects user history and preferences.

• The Flow Controller: Decides if the model needs a planning step or a retrieval call before generating the final answer.

The Result: The model receives a highly curated, coherent dataset designed specifically for reasoning—not a random blob of text.

6. How to Spec out Features Appropriately: The 4D Context Canvas

You now have the Context Pyramid (the categories of context) and the C.E.O. Framework (the engine to process it).

But when you sit down to design a specific feature, you need a tactical plan. You need to answer: What specific context does this feature require? Where does it live? And what happens when it’s missing?

This is the role of the 4D Context Canvas.

Most AI features do not fail because the model wasn’t smart enough. They fail because the team never explicitly defined the context required to succeed. They relied on dangerous assumptions like “we’ll just fetch that data” or “the model will figure it out.”

To prevent this, every AI feature must be stress-tested against four dimensions:

1. Demand: What is the model actually being asked to do?

2. Data: What context is required to get it right?

3. Discovery: How will we obtain that context reliably during runtime?

4. Defense: How will we detect failures and prevent incorrect outputs?

D1 — DEMAND

Defining the Model’s Job Precisely

If you cannot articulate the job, the model cannot perform it.

The primary cause of generic, erratic AI behavior is vague product requirements. Developers often write prompts without a clear specification, forcing the system to guess the user’s goal.

To fix this, you must translate fuzzy intent into a rigid Model Job Spec. You must clarify exactly what is being produced, for whom, and under what constraints.

The Transformation

• The Vague Request: “Draft a status update.”

• The Precise Demand: “Summarize key changes in Project X since the last report, formatted for Stakeholder Y, adopting the user’s concise tone, and strictly adhering to the weekly reporting schema.”

The Spec Checklist A proper demand spec must explicitly define:

1. Inputs: What raw materials will the model receive?

2. Assumptions: What acts as the “default” truth if data is missing?

3. Required Outputs: The exact format, length, and structure.

4. Success Criteria: What separates a “correct” answer from a “plausible” one?

D2 — DATA

Mapping the Hidden Dependencies

Once the job is defined, you must identify the fuel.

Every AI feature has hidden dependencies. The Data step forces you to map the exact context required to execute the job. This is best done via a Context Requirements Table—a simple audit that removes ambiguity.

The Context Audit For every piece of context, you must define four attributes:

1. The Signal: What entity, document, metric, or object is needed?

2. The Source: Where does it live? (SQL DB, Vector Store, API, Browser State).

3. Availability: Is this data always there, sometimes there (dependent on user history), or rarely there?

4. Sensitivity: Is this PII? Internal-only? Public?

Example: The AI Sprint Planner To plan a sprint, the model cannot just “know” agile. It needs:

• Backlog Items (Source: Jira API | Availability: High)

• Team Capacity (Source: HR System | Availability: Variable)

• Historical Velocity (Source: Analytics DB | Availability: High)

• Cross-Team Dependencies (Source: Knowledge Graph | Availability: Low/Hard to fetch)

When you map this honestly, you immediately see if your feature is viable or if you are missing the data pipelines to support it.

D3 — DISCOVERY

The Runtime Retrieval Strategy

Knowing what data you need is easy. Getting it in real-time is hard.

Discovery defines the logistics: how will the system locate, retrieve, and assemble the context at the exact moment the user presses “Enter”?

You must engineer a retrieval strategy that balances precision with latency:

The Retrieval Toolkit

• Search-Based: Using Vector Search for semantic themes or Keyword Search for exact matches (e.g., retrieving specific error codes).

• Graph-Based: Navigating the Knowledge Graph to find related entities (e.g., “Find the owner of the project that this ticket belongs to”).

• Precomputed: Using cached views or daily jobs to have heavy data ready instantly (e.g., “Yesterday’s sales totals”).

The Trade-off: Speed vs. Depth Teams must make hard choices here. Which context must be real-time? Which can be 24 hours old? Which data is too expensive to fetch for a sub-second interaction?

D4 — DEFENSE

Guardrails, Fallbacks, and Failure Modes

A feature is not ready until you have designed its failure state.

Defense is the layer that turns a fragile demo into a resilient production system. Because in the real world, APIs will time out, data will be stale, and models will hallucinate. Defense is about catching these failures before the user sees them.

The Four Defensive Layers

1. Pre-Checks (The Gatekeeper): Before invoking the model, ask: “Do we actually have enough context to answer this?” If the required entity is missing or the data is too old, block the generation and ask the user for clarification.

2. Post-Checks (The Auditor): After the model generates text, validate it. Does it match the JSON schema? Did it violate a safety policy? Is it logically consistent with the source data?

3. Fallback Paths (The Safety Net): When the system breaks, degrade gracefully. Do not show a stack trace.

   • Instead of a hallucination: Offer a conservative summary.

   • Instead of silence: Ask a clarifying question.

   • Instead of a crash: Revert to a hard-coded default.

4. Feedback Loops (The Teacher): Capture the signal from failure. Use explicit ratings (thumbs down) and implicit behavior (user editing the AI’s output) to detect patterns and patch the holes in your context logic.

   ---

   7. The Toolkit: Checklists, Templates, and Prompts

   You now have the theory and the architecture. This section provides the copy-paste assets you need to deploy them.

   Below is the Context Quality Checklist to run before every API call, and the Orchestrator Meta-Prompt that serves as the backbone for high-reasoning systems.

   Part 1: The Context Quality Checklist

   Use this “Pre-Flight Check” every single time before sending a payload to an LLM.

   Hallucinations are rarely model failures; they are almost always context failures. If you feed the model noise, ambiguity, or contradictions, no amount of “prompt engineering” will save you.

   Use this five-point audit to ensure the “brains” of your system are fed clean fuel.

   1. THE RELEVANCE AUDIT (Signal-to-Noise)

   • Curated: Does every piece of text in the context directly help answer the specific intent?

   • Distilled: Have you stripped away “kind of related” documents that dilute the model’s focus?

   • Cleaned: Did you remove decorative JSON metadata (internal IDs, unrelated flags) that wastes tokens and confuses the model?

   2. THE FRESHNESS AUDIT (Temporal Validity)

   • Timestamps: Are the documents and logs from the current relevant window?

   • State Awareness: Does the context reflect the system as it is right now (not cached from an hour ago)?

   • Deprecation: Have you filtered out “legacy” or “archived” entities that might mislead the model?

   3. THE SUFFICIENCY AUDIT (The “Missing Link” Check)

   • Entity Completeness: If the user asks about “Project X,” did you include Project X’s owner, status, and recent comments?

   • Dependency Mapping: Did you include related objects (e.g., the blocker preventing the project from moving)?

   • Gap Analysis: If a critical piece of data is missing, does the system know it is missing? (Preventing the model from guessing).

   4. THE STRUCTURE AUDIT (Parsability)

   • Segmentation: Is the context broken into clear, labeled sections (e.g., <History>, <Rules>, <Data>)?

   • Explicit Relationships: Are connections described explicitly (”X blocks Y”) rather than implied?

   • Schema Adherence: Is domain knowledge presented as structured data, not a dump of raw text?

   5. THE CONSTRAINT AUDIT (Safety & Business Logic)

   • Hard Rules: Are business rules embedded as commands, not suggestions?

   • Tone & Style: Are formatting and voice requirements explicitly defined?

   • Permissions: Is the model explicitly told what it cannot do or access?

   Part 2: The Orchestrator Meta-Prompt

   The Template for High-Fidelity Reasoning

   This is the template used by top-tier AI agents. It forces the LLM to operate within a “structured reasoning cage,” dramatically reducing drift.

   Copy this structure for your system prompt:

[System Instructions] 
You are an AI assistant operating inside a structured context engine. 
Your primary directive is to follow all business rules, domain constraints, and formatting instructions exactly. 
You must not invent facts. You are restricted to the context provided below.

[User Intent]
{inferred_intent}   <-- The interpreted goal, not just the raw text
{explicit_prompt}   <-- What the user actually typed

[Relevant Entities]
{structured_entities} <-- The specific nouns involved (tickets, users, files)

[Relationships]
{entity_relationships} <-- How those nouns connect (dependencies, ownership)

[Session State]
{recent_messages}     <-- Short-term memory
{recent_selections}   <-- Implicit context (clicks, hovers)

[User Profile]
{role}                <-- Who is asking?
{tone_preferences}    <-- How do they like to read?
{prior_examples}      <-- Few-shot examples of success

[Domain Context]
{retrieved_docs}      <-- RAG results, strictly filtered
{summaries}           <-- Distilled knowledge
{attached_metadata}   <-- Environment variables (Time, Location)

[Rules & Constraints]
{business_rules}      <-- Hard logic (e.g., “Never promise dates”)
{policies}            <-- Safety guidelines
{prohibited_actions}  <-- Negative constraints

[Environment Signals]
{calendar_events}     <-- Temporal context
{deadlines}           <-- Urgency signals
{system_status}       <-- Operational reality

[Task Instructions]
Clear, step-by-step instructions for what the model must produce right now.

[Output Schema]
{json_schema_or_output_structure} <-- The exact format required

Adopting this template typically reduces hallucination rates by over 70% in production systems.

Part 3: The Template in Action (Example)

Scenario: The AI Product Management Assistant

To show you exactly how this looks populated, here is a real-world example of an orchestrator prompt generating a Weekly Status Update.

Notice how the “raw” data has been transformed into structured knowledge before the model ever sees it.

THE PROMPT PAYLOAD:

[System Instructions]
You are an AI assistant operating inside a structured context engine for a product team.

You write weekly product status updates for senior stakeholders (VP Product, CTO, CEO) based strictly on the context provided below.

You must:
- Follow all business rules, domain constraints, and formatting instructions exactly.
- Never invent projects, metrics, incidents, or timelines that are not explicitly present in Domain Context.
- Treat the Domain Context and Rules sections as the single source of truth.
- If critical information is missing, state what is missing. Do not guess.

[User Intent]
{inferred_intent}:
“Summarize the most important product changes, progress, risks, and next steps for the past week into an executive-ready weekly update.”
{explicit_prompt}:
“Can you draft this week’s product update for leadership based on what changed since last Monday?”

[Relevant Entities]
{structured_entities}:
- project_roadmap_item:
    id: “PRJ-142”
    title: “Onboarding Funnel Revamp”
    owner: “Sara”
    status: “In Progress”
    target_release: “2025-12-01”
- project_roadmap_item:
    id: “PRJ-087”
    title: “AI Assistant v2”
    owner: “Imran”
    status: “Shipped”
    target_release: “2025-11-15”
- metric:
    id: “MTR-DAU”
    name: “Daily Active Users”
    current_value: 18240
    previous_value: 17680
    unit: “users”
- incident:
    id: “INC-221”
    title: “Checkout Latency Spike”
    status: “Resolved”
    severity: “High”

[Relationships]
{entity_relationships}:
- “PRJ-142” depends_on “PRJ-087”
- “INC-221” impacted “checkout_conversion”
- “MTR-DAU” improved_after “AI Assistant v2” release
- “PRJ-087” linked_to_release “2025.11.15-prod”

[Session State]
{recent_messages}:
- 2025-11-17T09:03Z – User: “Last week’s update is in the doc; I want something similar but shorter.”
- 2025-11-17T09:04Z – Assistant: “Understood, I will keep a similar structure but be more concise.”
- 2025-11-17T09:06Z – User: “Don’t oversell wins; keep it realistic.”

{recent_selections}:
- User highlighted last week’s “Risks & Blockers” section.
- User opened the “AI Assistant v2 – Launch Notes” document.

[User Profile]
{role}:
- “Director of Product, responsible for AI & Growth initiatives.”

{tone_preferences}:
- Confident but not hype.
- Data-informed, not overly narrative.
- Clear separation of “What happened”, “Why it matters”, and “What’s next”.

{prior_examples}:
- Example snippet: “This week we completed the rollout of the new onboarding experiment to 50% of new users. Early results show a +3.2% lift in activation.”

[Domain Context]
{retrieved_docs}:
- “Weekly Update – 2025-11-10” (last week’s product update)
- “AI Assistant v2 – Launch Notes”
- “Onboarding Funnel – Experiment Spec v3”
- “Incident Report – INC-221 Checkout Latency”

{summaries}:
- Last Week’s Update Summary: “Focused on preparing AI Assistant v2 launch, mitigating checkout latency incidents.”
- AI Assistant v2 Launch Notes Summary: “Shipped on 2025-11-15 to 100% of users, goals: improve task completion speed.”
- Incident INC-221 Summary: “High-severity latency issue, resolved within 4 hours, root cause: misconfigured database index.”

{attached_metadata}:
- current_week_range: “2025-11-10 to 2025-11-17”
- timezone: “America/Los_Angeles”
- environment: “Production”

[Rules & Constraints]
{business_rules}:
- Do not share internal incident IDs (e.g., INC-221); describe incidents in business terms.
- Do not reference customers by name; aggregate or anonymize.
- Always tie product work back to business outcomes (activation, retention).

{policies}:
- No forward-looking commitments beyond the existing roadmap.
- Maintain consistency with official metric definitions.

{formatting_requirements}:
- Structure exactly as: 1. Highlights, 2. Metrics & Impact, 3. Risks & Blockers, 4. Next Week.
- Use Markdown headers (H2).
- Total length under 600 words.

{prohibited_actions}:
- Do not fabricate metrics, dates, or launches.
- Do not mention features not explicitly referenced in Domain Context.

[Environment Signals]
{calendar_events}:
- Today is Monday, 2025-11-17.
- “Exec Product Sync” is scheduled for 15:30 today.

{deadlines}:
- Q4 goals lock on 2025-12-01.

{system_status}:
- All systems operational. Analytics data is fresh as of 08:00Z.

[Task Instructions]
Using only the information provided above:
1. Draft a weekly product update that is realistic, grounded, and aligned with the user’s tone.
2. Follow the required structure: Highlights, Metrics & Impact, Risks & Blockers, Next Week.
3. Emphasize what actually changed this week compared to last week.
4. Connect product work to business outcomes using the provided metrics.
5. If there are known gaps (e.g., missing metric results), note them transparently.

[Output Schema]
Return the final result as a JSON object with the following shape:
{
  “highlights_markdown”: “string – Markdown-formatted section”,
  “metrics_and_impact_markdown”: “string – Markdown-formatted section”,
  “risks_and_blockers_markdown”: “string – Markdown-formatted section”,
  “next_week_markdown”: “string – Markdown-formatted section”,
  “notes_for_user”: “string – Any caveats or assumptions made”
}

Final Words: The Age of Context

We are crossing the threshold into what will be remembered as The Age of Context.

In the early 2020s, the industry was obsessed with “Model-First” development. We built wrappers around APIs, relied on clever prompts, and chased demo-day magic. Those tools dazzled for a moment, but they inevitably disappointed the second they encountered the messy reality of enterprise work. Those systems are fading, and they should.

The next decade belongs to Context-First architectures.

We are moving toward systems that do not just generate text, but that understand users with the intuition of a long-time colleague. Systems that navigate institutional history like a veteran employee. Systems that anticipate next steps like a strategic partner. And systems that respect boundaries with the rigor of a compliance officer.

The Only Durable Moat Crucially, this is where the business value lies. A frontier model is a commodity; anyone can rent the smartest brain in the world for a few cents per token. But Context is an asset you own. It is the only defensible moat left.

• Your domain knowledge graph is a moat.

• Your codified business rules are a moat.

• Your user memory bank is a moat.

• Your operational workflows are a moat.

• Your environmental sensors are a moat.

• Your orchestration logic is a moat.

• Your data provenance is a moat.

And these moats compound.

The teams that embrace context engineering today will build products that feel impossibly intelligent—not because they have a better model than their competitors, but because they have architected a system where intelligence is distributed across layers of memory, structure, reasoning, and constraint.

If you are building AI products, this is your invitation—and your responsibility.

Stop waiting for the next model breakthrough to solve your problems.

The true foundation of intelligence is not inference. It is Context.

It’s okay if you don’t know what they are, as this article is primarily intended to explain everything about vector databases in detail.

But given how popular they have become lately, I think it is crucial to be aware of what makes them so powerful that they gained so much popularity, and their practical utility not just in LLMs but in other applications as well.

Let’s dive in!

What are vector databases?

Objective

To begin, we must note that vector databases are NOT new.

In fact, they have existed for a pretty long time now. You have been indirectly interacting with them daily, even before they became widely popular lately. These include applications like recommendation systems, and search engines, for instance.

Simply put, a vector database stores unstructured data (text, images, audio, video, etc.) in the form of vector embeddings.

Each data point, whether a word, a document, an image, or any other entity, is transformed into a numerical vector using ML techniques (which we shall see ahead).

This numerical vector is called an embedding, and the model is trained in such a way that these vectors capture the essential features and characteristics of the underlying data.

Considering word embeddings, for instance, we may discover that in the embedding space, the embeddings of fruits are found close to each other, which cities form another cluster, and so on.

This shows that embeddings can learn the semantic characteristics of entities they represent (provided they are trained appropriately).

Once stored in a vector database, we can retrieve original objects that are similar to the query we wish to run on our unstructured data.

In other words, encoding unstructured data allows us to run many sophisticated operations like similarity search, clustering, and classification over it, which otherwise is difficult with traditional databases.

To exemplify, when an e-commerce website provides recommendations for similar items or searches for a product based on the input query, we’re (in most cases) interacting with vector databases behind the scenes.

Before we get into the technical details, let me give you a couple of intuitive examples to understand vector databases and their immense utility.

Example #1

Let's imagine we have a collection of photographs from various vacations we’ve taken over the years. Each photo captures different scenes, such as beaches, mountains, cities, and forests.

Now, we want to organize these photos in a way that makes it easier to find similar ones quickly.

Traditionally, we might organize them by the date they were taken or the location where they were shot.

However, we can take a more sophisticated approach by encoding them as vectors.

More specifically, instead of relying solely on dates or locations, we could represent each photo as a set of numerical vectors that capture the essence of the image.

💡 While Google Photos doesn't explicitly disclose the exact technical details of its backend systems, I speculate that it uses a vector database to facilitate its image search and organization features, which you may have already used many times.

Let’s say we use an algorithm that converts each photo into a vector based on its color composition, prominent shapes, textures, people, etc.

Each photo is now represented as a point in a multi-dimensional space, where the dimensions correspond to different visual features and elements in the image.

Now, when we want to find similar photos, say, based on our input text query, we encode the text query into a vector and compare it with image vectors.

Photos that match the query are expected to have vectors that are close together in this multi-dimensional space.

Suppose we wish to find images of mountains.

In that case, we can quickly find such photos by querying the vector database for images close to the vector representing the input query.

A point to note here is that a vector database is NOT just a database to keep track of embeddings.

Instead, it maintains both the embeddings and the raw data which generated those embeddings.

Why is that necessary, you may wonder?

Considering the above image retrieval task again, if our vector database is only composed of vectors, we would also need a way to reconstruct the image because that is what the end-user needs.

When a user queries for images of mountains, they would receive a list of vectors representing similar images, but without the actual images.

By storing both the embeddings (the vectors representing the images) and the raw image data, the vector database ensures that when a user queries for similar images, it not only returns the closest matching vectors but also provides access to the original images.

Example #2

In this example, consider an all-text unstructured data, say thousands of news articles, and we wish to search for an answer from that data.

Traditional search methods rely on exact keyword search, which is entirely a brute-force approach and does not consider the inherent complexity of text data.

In other words, languages are incredibly nuanced, and each language provides various ways to express the same idea or ask the same question.

For instance, a simple inquiry like "What's the weather like today?" can be phrased in numerous ways, such as "How's the weather today?", "Is it sunny outside?", or "What are the current weather conditions?".

This linguistic diversity makes traditional keyword-based search methods inadequate.

As you may have already guessed, representing this data as vectors can be pretty helpful in this situation too.

Instead of relying solely on keywords and following a brute-force search, we can first represent text data in a high-dimensional vector space and store them in a vector database.

When users pose queries, the vector database can compare the vector representation of the query with that of the text data, even if they don't share the exact same wording.

How to generate embeddings?

At this point, if you are wondering how do we even transform words (strings) into vectors (a list of numbers), let me explain.

To build models for language-oriented tasks, it is crucial to generate numerical representations (or vectors) for words.

This allows words to be processed and manipulated mathematically and perform various computational operations on words.

The objective of embeddings is to capture semantic and syntactic relationships between words. This helps machines understand and reason about language more effectively.

In the pre-Transformers era, this was primarily done using pre-trained static embeddings.

Essentially, someone would train embeddings on, say, 100k, or 200k common words using deep learning techniques and open-source them.

Consequently, other researchers would utilize those embeddings in their projects.

The most popular models at that time (around 2013-2017) were:

• Glove

• Word2Vec

• FastText, etc.

These embeddings genuinely showed some promising results in learning the relationships between words.

For instance, at that time, an experiment showed that the vector operation (King - Man) + Woman returned a vector near the word Queen.

That’s pretty interesting, isn’t it?

In fact, the following relationships were also found to be true:

• Paris - France + Italy ≈ Rome

• Summer - Hot + Cold ≈ Winter

• Actor - Man + Woman ≈ Actress

• and more.

So, while these embeddings captured relative word representations, there was a major limitation.

Consider the following two sentences:

• Convert this data into a table in Excel.

• Put this bottle on the table.

Here, the word “table” conveys two entirely different meanings:

• The first sentence refers to a “data” specific sense of the word “table.”

• The second sentence refers to a “furniture” specific sense of the word “table.”

Yet, static embedding models assigned them the same representation.

Thus, these embeddings didn’t consider that a word may have different usages in different contexts.

But this was addressed in the Transformer era, which resulted in contextualized embedding models powered by Transformers, such as:

• BERT: A language model trained using two techniques:

  • Masked Language Modeling (MLM): Predict a missing word in the sentence, given the surrounding words.

  • Next Sentence Prediction (NSP).

  • We shall discuss it in a bit more detail shortly.

• DistilBERT: A simple, effective, and lighter version of BERT, which is around 40% smaller:

  • Utilizes a common machine learning strategy called student-teacher theory.

  • Here, the student is the distilled version of BERT, and the teacher is the original BERT model.

  • The student model is supposed to replicate the teacher model’s behavior.

• SentenceTransformer: If you read the most recent deep dive on building classification models on ordinal data, we discussed this model there.

  • Essentially, the SentenceTransformer model takes an entire sentence and generates an embedding for that sentence.

• This differs from the BERT and DistilBERT models, which produce an embedding for all words in the sentence.

There are more models, but we won't go into more detail here, and I hope you get the point.

The idea is that these models are quite capable of generating context-aware representations, thanks to their self-attention mechanism and appropriate training mechanism.

BERT

For instance, if we consider BERT again, we discussed above that it uses the masked language modeling (MLM) technique and next sentence prediction (NSP).

These steps are also called the pre-training step of BERT because they involve training the model on a large corpus of text data before fine-tuning it on specific downstream tasks.

Pre-training, in the context of machine learning model training, refers to the initial phase of training where the model learns general language representations from a large corpus of text data. The goal of pre-training is to enable the model to capture the syntactic and semantic properties of language, such as grammar, context, and relationships between words. While the text itself is unlabeled, MLM and NSP are two tasks that help us train the model in a supervised fashion. Once the model is trained, we can use the language understanding capabilities that the model acquired from the pre-training phase, and fine-tune the model on task-specific data.

Moving on, let’s see how the pre-training objectives of masked language modeling (MLM) and next sentence prediction (NSP) help BERT generate embeddings.

#1) Masked Language Modeling (MLM)

• In MLM, BERT is trained to predict missing words in a sentence. To do this, a certain percentage of words in most (not all) sentences are randomly replaced with a special token, [MASK].

• BERT then processes the masked sentence bidirectionally, meaning it considers both the left and right context of each masked word, that is why the name “Bidirectional Encoder Representation from Transformers (BERT).”

• For each masked word, BERT predicts what the original word is supposed to be from its context. It does this by assigning a probability distribution over the entire vocabulary and selecting the word with the highest probability as the predicted word.

• During training, BERT is optimized to minimize the difference between the predicted words and the actual masked words, using techniques like cross-entropy loss.

#2) Next Sentence Prediction (NSP)

• In NSP, BERT is trained to determine whether two input sentences appear consecutively in a document or whether they are randomly paired sentences from different documents.

• During training, BERT receives pairs of sentences as input. Half of these pairs are consecutive sentences from the same document (positive examples), and the other half are randomly paired sentences from different documents (negative examples).

• BERT then learns to predict whether the second sentence follows the first sentence in the original document (label 1) or whether it is a randomly paired sentence (label 0).

• Similar to MLM, BERT is optimized to minimize the difference between the predicted labels and the actual labels, using techniques like binary cross-entropy loss.

If we look back to MLM and NSP, in both cases, we did not need a labeled dataset to begin with. Instead, we used the structure of the text itself to create the training examples. This allows us to leverage large amounts of unlabeled text data, which is often more readily available than labeled data.

Now, let's see how these pre-training objectives help BERT generate embeddings:

• MLM: By predicting masked words based on their context, BERT learns to capture the meaning and context of each word in a sentence. The embeddings generated by BERT reflect not just the individual meanings of words but also their relationships with surrounding words in the sentence.

• NSP: By determining whether sentences are consecutive or not, BERT learns to understand the relationship between different sentences in a document. This helps BERT generate embeddings that capture not just the meaning of individual sentences but also the broader context of a document or text passage.

With consistent training, the model learns how different words relate to each other in sentences. It learns which words often come together and how they fit into the overall meaning of a sentence.

This learning process helps BERT create embeddings for words and sentences, which are contextualized, unlike earlier embeddings like Glove and Word2Vec:

Contextualized means that the embedding model can dynamically generate embeddings for a word based on the context they were used in.

As a result, if a word would appear in a different context, the model would return a different representation.

This is precisely depicted in the image below for different uses of the word Bank.

For visualization purposes, the embeddings have been projected into 2d space using t-SNE.

As depicted above, the static embedding models — Glove and Word2Vec produce the same embedding for different usages of a word.

However, contextualized embedding models don’t.

In fact, contextualized embeddings understand the different meanings/senses of the word “Bank”:

• A financial institution

• Sloping land

• A Long Ridge, and more.

As a result, these contextualized embedding models address the major limitations of static embedding models.

The point of the above discussion is that modern embedding models are quite proficient at the encoding task.

As a result, they can easily transform documents, paragraphs, or sentences into a numerical vector that captures its semantic meaning and context.

In an era of voice-driven interfaces and AI-powered assistants, extracting meaningful insights from audio is more important than ever. Whether you're a developer, product builder, or AI tinkerer, this guide will show you how to build a full-fledged Audio Analysis Toolkit that can:

• Transcribe audio with speaker labels and timestamps

• Summarize long conversations

• Detect sentiment and emotional tone

• Extract key topics

• Enable natural-language Q&A on audio content

The best part? You can use it either as:

1. A beautiful web UI powered by Streamlit, OR

2. A backend MCP server connected to Claude via Cursor for agentic interaction.

Let’s dive into both modes of using the toolkit.

Tech Stack Overview

• AssemblyAI – API for audio transcription and advanced analysis (sentiment, topics, summarization)

• Streamlit – Rapid frontend for building interactive UI

• Cursor MCP (Model Context Protocol) – Interface for agents like Claude to call backend functions

• Claude (via Cursor) – Agent that can consume your toolkit through natural language

• Python 3.12+

Here's our workflow:

• User's audio input is sent to AssemblyAI via a local MCP server.

• ​AssemblyAI​ transcribes it while providing the summary, speaker labels, sentiment, and topics.

• Post-transcription, the user can also chat with audio.

Dual Tool Structure: Transcription + Audio Analysis (via MCP)

Transcription MCP tool

This tool accepts an audio input from the user and transcribes it using AssemblyAI.

We also store the full transcript to use in the next tool.

Audio analysis tool

Next, we have a tool that returns specific insights from the transcript, like speaker labels, sentiment, topics, and summary.

Based on the user’s input query, the corresponding flags will be automatically set to True when the Agent will prepare the tool call via MCP:

Bonus: It’s RAG-Ready

Once transcribed, the transcript can even be indexed into a vector database or used in an LLM context window for true RAG-style audio Q&A, especially useful for:

• Meeting assistants

• Interview breakdowns

• Legal depositions

• Customer call analysis

Let’s implement this!

Option 1: Use as a Streamlit Web App 

This mode is perfect if you want a user-friendly, visually rich interface for uploading and analyzing audio files.

This web-based interface is powered by Streamlit and comes with:

• Audio file upload: .mp3, .wav, .mp4, etc.

• Full transcript with timestamps

• Speaker-wise breakdown

• Sentiment visualized with emojis and colors

• AI-generated summaries

• Top topics and relevance

• Ask questions about the audio in a chatbot-style interface

How to Run It? 

1. Clone the repo and set up the environment.

git clone https://github.com/edityeah/ai-hub.git
cd audio-analysis-toolkit
python -m venv .venv
source .venv/bin/activate    # Or .venvScriptsactivate on Windows
pip install -r requirements.txt	

2. Add your AssemblyAI API Key

Create a .env file:

ASSEMBLYAI_API_KEY=your_key_here		

Run the app

streamlit run app.py		

Option 2: Use as a Cursor MCP Server (for Claude Agents)

If you prefer working agentically — that is, using an AI like Claude to invoke capabilities — you can run this toolkit as an MCP server.

What is MCP?

MCP (Model Context Protocol) is how tools can plug into Cursor and be used by agents like Claude. It allows your local Python scripts to respond to natural language instructions inside the Cursor IDE.

Setting It Up:

1. Ensure server.py is in your project directory (this script exposes transcription logic via HTTP or CLI).

2. Configure Cursor MCP:
   In Cursor → Preferences → MCP Servers → Global, add:

{
  "mcpServers": {
    "assemblyai": {
      "command": "python",
      "args": ["server.py"],
      "env": {
        "ASSEMBLYAI_API_KEY": "your_key_here"
      }
    }
  }
}			

3. Open the project folder in Cursor
Cursor will automatically launch the MCP server if the config and working directory are correct.

4. Call it from Claude:
Inside Cursor, invoke the tool like this:

@assemblyai: transcribe "path/to/audio.mp3"		

Claude will now interact with your audio analysis server in real-time and return intelligent answers.

Final Thoughts

With just a few lines of Python and AssemblyAI’s APIs, you can build a full-featured audio intelligence system. Whether you want to offer this as a product, internal tool, or agentic function inside your AI IDE, this toolkit gives you the flexibility to do both.

💡 Ready to build? Try both modes and see what fits your workflow best!

In today's organizations, data is scattered across countless platforms—Slack, Gmail, SQL databases, Google Drive, GitHub, and more. Bringing these data sources into one unified, queryable system is often complex, requiring expensive data integration platforms or risky cloud-based services.

I've solved this by building a 100% local MCP (Model Context Protocol) server that can connect to 200+ data sources and enable natural language querying across them via a simple chat interface.

Let’s walk you through how I've built this powerful federated query system using MindsDB, Cursor IDE, and Docker.

The Unified MCP Server

This project revolves around building a local-first, modular, and scalable MCP server that:

• Connects to over 200+ data sources.

• Allows natural language querying across structured and unstructured platforms.

• Is completely self-hosted, ensuring maximum data privacy and security.

I've used:

• MindsDB: A federated query engine that understands complex, multi-source queries.

• Cursor IDE: The interface that hosts the MCP chat and connects to the server.

• Docker: For local containerization and seamless deployment.

Key Features

FeatureDescription🔍 Multi-source QueryingQuery 200+ data sources with a single natural language prompt.🧠 Context-Aware AIAI responses tailored to your connected organizational data.💬 Real-time Chat InterfaceUnified chat window powered by Cursor IDE.🔧 Modular DesignEasily plug in new data sources via MindsDB integrations.🐳 Docker-based DeploymentFully local deployment ensuring security and control.🔒 Privacy-firstData remains on your local machines—no cloud dependency.

How It Works: Architecture Flow

1. User submits a query via Cursor’s chat interface.

2. Cursor client forwards the query to the MCP server.

3. MCP server sends the query to MindsDB’s federated query engine.

4. MindsDB retrieves results from the relevant connected data sources.

5. MCP server receives results and passes them back to Cursor.

6. Cursor displays the AI-generated response in the chat interface.

Step-by-Step Implementation Details

Prerequisites:

• Docker

• Cursor IDE

• MindsDB (installed locally)

Docker Setup

MindsDB provides a pre-built Docker image for quick setup.

To install MindsDB locally, I ran the following command:

docker run --name mindsdb_container -e MINDSDB_APIS="http,mcp" 
-p 47334:47334 -p 47337:47337 mindsdb/mindsdb		

🖥️ Start MindsDB GUI

After installing, go to  127.0.0.1:47334 in your browser to access the MindsDB editor.

This GUI allows me to:

• Connect over 200+ data sources

• Run SQL queries against them

• Manage the federated query engine visually

Integrate Data Sources

Let's start building our federated query engine by connecting our data sources to MindsDB.

We use Slack, Gmail, GitHub, and Hacker News as our federated data sources.

Integrate MCP Server with Cursor

After building the federated query engine, let's unify our data sources by connecting them to MindsDB's MCP server.

Go to: File → Preferences → Cursor Settings → MCP → Add new global MCP server

In the JSON file, add the following 👇

Done! Our MindsDB MCP server is live and connected to Cursor!

The MCP server offers two tools:

• list_databases: Lists all data sources connected to MindsDB.

• query: Answers user queries on the federated data.

Apart from Claude and Cursor, the MindsDB MCP server also works with the new OpenAI MCP integration:

The usage of list_databases is depicted below. We asked it to list all the data sources it can access and retrieved the four data sources we connected earlier:

Below is another usage, where we asked it to list the available Slack channels.

While this is quite basic, I have shown a much more sophisticated usage in the video attached at the top.

• I fetched info from Hacker News.

• I generated a nicely formatted summary of the fetched details and sent that as a message to Slack.

Chatting With My Data

I can now submit queries directly via Cursor’s chat interface:

• “Show me the last five emails from Gmail.”

• “Summarize recent Slack mentions.”

• “Fetch the top stories from Hacker News.”

• “List all open GitHub issues in my repository.”

Why I Built This

• Data Privacy: Fully local, no cloud dependency.

• Unified Access: Query Slack, Gmail, GitHub, and more from one place.

• Scalable: Easy to add new data sources.

• Simple Interface: Natural language queries—no SQL skills required.

💡 Closing Thoughts

I built this unified MCP server to make data querying seamless, private, and conversational.

It’s:

• Local-first for security.

• Modular for flexibility.

• Simple for everyday use.

If you’re looking to own your data workflow, this is a powerful, scalable approach.

Lately, there’s been a lot of buzz around Model Context Protocol (MCP). You’ve probably heard of it.

Today, let’s break it down.

Intuitively speaking, MCP is like a USB-C port for your AI applications.

Just as USB-C offers a standardized way to connect devices to various accessories, MCP standardizes how AI apps connect to different data sources and tools.

Let’s dive in a bit more technically.

What is MCP?

At its core, MCP follows a client-server architecture, where a host application can connect to multiple servers.

It has three key components:

• Host

• Client

• Server

Here’s a quick overview before we go deeper 👇

• Host: Represents any AI app (like Claude Desktop or Cursor) that provides an environment for AI interactions, accesses tools and data, and runs the MCP Client.

• MCP Client: Operates within the host and enables communication with MCP servers.

• MCP Server: Exposes specific capabilities and provides access to data like:

  • Tools: Let LLMs perform actions via your server

  • Resources: Share data/content from your server with LLMs

  • Prompts: Provide reusable prompt templates and workflows

How Does the Client-Server Communication Work?

Understanding this exchange is key to building your own MCP client-server setup.

Let’s look at the flow:

1. Capability Exchange:

   • The client sends an initial request to learn the server’s capabilities.

   • The server responds with a list of its tools, resources, prompt templates, and other features.

   📦 Example: A Weather API server may reply with the tools it offers (like “getTemperature”), available prompt workflows, and supported parameters.

2. Connection Acknowledgment:

   • The client confirms successful connection.

   • All further interactions build on this shared understanding.

Why Is This Powerful?

Here’s a common pain point with traditional APIs:

• If your API initially requires two parameters (e.g., location and date for a weather service), users integrate their applications to send requests with those exact parameters.

• Later, if you decide to add a third required parameter (e.g., unit for temperature units like Celsius or Fahrenheit), the API’s contract changes.

• This means all users of your API must update their code to include the new parameter. If they don’t update, their requests might fail, return errors, or provide incomplete results.

How MCP Solves This?

MCP introduces a dynamic and flexible approach that contrasts sharply with traditional APIs.

• For instance, when a client (e.g., an AI application like Claude Desktop) connects to an MCP server (e.g., your weather service), it sends an initial request to learn the server’s capabilities.

• The server responds with details about its available tools, resources, prompts, and parameters. For example, if your weather API initially supports location and date, the server communicates these as part of its capabilities.

• If you later add a unit parameter, the MCP server can dynamically update its capability description during the next exchange. The client doesn’t need to hardcode or predefine the parameters—it simply queries the server’s current capabilities and adapts accordingly.

• This way, the client can then adjust its behavior on-the-fly, using the updated capabilities (e.g., including unit in its requests) without needing to rewrite or redeploy code.

Wrapping Up

I hope this clarifies what MCP does and how it compares to traditional APIs.

Do you think MCP is more powerful than traditional API setups?

In upcoming posts, we’ll explore how to build custom MCP servers and share hands-on demos.

Stay tuned!

Product Discovery is by far the most critical area for a Product Manager.

But, it is largely misunderstood. Teams waste time and energy delivering ideas that fail to drive the outcomes they expect.

Too often, teams spend months building features that fail to meet user needs or drive business impact. If you’ve ever launched something that barely got used, you already know why Product Discovery matters.

In this Product Discovery 101 guide, you’ll learn:

• What Product Discovery is

• Why Product Discovery is important

• The difference between Continuous and Initial Product Discovery

• How to apply Product Discovery: techniques, frameworks, and tools

• Pros and cons of Jira Product Discovery

• Where to find Product Discovery courses, templates, and resources

1. Why do We Need Product Discovery?

1.1 Managing risks

Let me tell you a secret: In product management, most ideas won’t work as expected. As Marty Cagan says in Inspired:

„The first truth is that at least half of your ideas are just not going to work”

Good teams assume at least three-quarters of their ideas won’t perform as they hope.

But why does it happen?

Product management is, at its heart, about managing risk. And for every product, 5 essential risks can materialize:

• Value Risk: Will this idea truly create value for customers?

• Usability Risk: Will users figure out how to use it?

• Viability Risk: Can our business support it?

• Feasibility Risk: Can we build it with our current technology?

• Ethical Risk: Should we build it at all? Are there ethical considerations?

Addressing these questions before you code helps avoid the costly trap of building the wrong thing.

1.2 Learning by delivering

Many teams adopt Agile frameworks to deliver software iteratively and adapt quickly.

For example, in Scrum, we have a Sprint Review. During this workshop, we discuss the progress with the stakeholders and decide how to adapt:

But what will happen if we throw random ideas into the Product Backlog?

Some might work, but most won’t. This is "learning by delivering." Unfortunately, this approach results in waste and rework.

Some might say it’s not a problem, as the Sprint is often 1-2 weeks long.

But this will happen every Sprint. Every Sprint, the product team selects ideas, implements them, and delivers a production-ready increment to discover those were not good ideas.

And the best ideas might not even be on the list.

So, we’d like to understand:

• How can we come up with better ideas?

• How can we test those ideas before the implementation?

And the answer is Product Discovery.

2. What is Continuous Product Discovery?

Continuous Product Discovery is a special type of Product Discovery for an existing product. It’s all about regularly exploring opportunities and identifying and testing your assumptions.

Instead of a single, upfront research phase, you keep the discovery going in parallel with delivery.

2.1 Continuous Discovery and Continuous Delivery

You might have heard about Dual-Track Development or Dual-Track Agile. It simply means having two streams running in parallel:

• Continuous Discovery: Discover and validate what product to build.

• Continuous Delivery: Implement and deliver the validated product to the market

What's important here is that this is not a waterfall. Some team members talk to the customers, explore people's problems, brainstorm solutions, identify assumptions, and perform experiments. At the same time, the team implements and delivers validated ideas to the market.

By testing ideas early, you dramatically reduce waste and rework. Product Discovery results in a validated Product Backlog. In particular, high-risk assumptions are tested upfront.

2.2 The Product Trio

There is a common misconception. Some say that the Product Manager decides what to build, and Engineers and designers should focus on how to build it.

Have you heard that before?

It hurts my ears because Product Discovery is not a task for a single person. I deeply believe that instead of building those silos and stage gates, we should embrace a collaborative approach.

In Continuous Discovery Habits, Teresa Torres introduced the concept of a Product Trio: Product Manager, Product Designer, and Engineer working together. This will help you build a shared understanding and stay open to different perspectives.

In particular, I have repeatedly found that the best ideas often come from my engineers. They know what’s technically possible.

Tip: Please remember that the Product Trio is not a rigid framework. It’s the default recommendation that can expand and contract. For example, you might include a Product Marketing Manager or exclude a Product Designer for an API product.

2.3 Inside Continuous Product Discovery

Think of Continuous Product Discovery as a series of iterative cycles, often visualized with the Double Diamond model:

Tip: Critically, Product Discovery is not a linear process. For example, you can loop back at any time if an experiment fails or you need more research.

Here, we explore the problem space to understand and define opportunities (problems, needs, desires) that, when solved, will drive the desired outcomes.

The two stages of divergent and convergent thinking:

• Stage 1: Explore

  • Performing customer interviews (e.g., weekly)

  • Interviewing stakeholders

  • Learning from usage analytics (collecting and analyzing data on how users interact with a product or service, e.g., Pendo.io)

  • Exploring data analytics (analyzing large datasets to uncover trends, patterns, and relationships, e.g., Microsoft Power BI)

  • Leveraging other sources of insights in Product Discovery: surveys, market trends, SEO, and SEM reporting

• Stage 2: Define

  • Mapping opportunities (e.g., Opportunity Solution Tree by Teresa Torres)

  • Prioritizing opportunities (e.g., Opportunity Score by Dan Olsen, ICE score)

The Right Diamond

Here, we explore the solution space to identify the right ideas to build.

The two stages of divergent and convergent thinking:

• Stage 3: Ideate

  • Brainstorming solutions to identified opportunities

  • Formulating testable assumptions related to those ideas

• Stage 4: Test

  • Perform experiments to prove or disprove identified assumptions

Continuous Product Discovery Example

In this example, we started with the desired outcome of increasing customer engagement for a video streaming platform by 20%. The team decided to map their steps with the Opportunity Solution Tree.

Stage 1, Stage 2: Explore and Define

After interviewing customers, analyzing product analytics, and talking to support and customer success, the Product Trio identified opportunities and child opportunities related to the desired outcome, e.g.:

• I can’t find anything to watch,

• I can’t figure out how to search for a specific show).

Next, they prioritized opportunities that are important for the users, where their satisfaction with their ability to achieve a specific outcome was low.

Stage 3: Ideate

After prioritizing opportunities, everyone on the Product Trio brainstormed individually. The team members met to discuss their ideas.

They immediately rejected a few ideas for one of the opportunities (“I can’t figure out how to search for a specific show”). The two most promising ideas left were:

• Displaying a search box,

• Displaying recommended shows.

Stage 4: Test

The team used User Story Mapping to identify assumptions related to value, usability, and feasibility. They identified some as risky and planned experiments to test those assumptions.

For example, they created a user prototype in Figma to test whether users understand how a search box works. They gave selected users a task to accomplish, tracked every user's click, and assumed the assumption was correct if at least 70% of the users would complete their task on the first attempt.

The test was successful, so there was no need to consider alternative solutions or talk to the users again.

Previously, the Product Manager ensured with the stakeholders the the solution would also work for the business (she talked to CSM, Marketing, and Sales).

A few days later, the engineers selected this idea for implementation.

3. What is Initial Product Discovery?

Initial Product Discovery comes into play when creating a new product.

In this phase, you define your Product Vision, draft a Business Model, and identify the right market. You’re not worrying yet about continuous iteration.

You’re trying to figure out if there’s a market at all.

Initial Product Discovery follows a very similar workflow with different activities possible in each stage:

Again, there is no “Product Discovery Process,” but rather a series of iterative cycles.

4. Jira Product Discovery: Pros & Cons

If you work with Jira, you might have heard about Jira Product Discovery. It offers some handy features like a Now-Next-Later roadmap, which is a perfect choice for visualizing big-picture plans:

But JIRA Product Discovery misses the critical aspects of Continuous Product Discovery and encourages some bad practices:

• No built-in way to map opportunities

• No straightforward way to plan experiments

Some try to hack it, but I have a strong opinion: it’s not a Continuous Product Discovery tool. You will do much better creating the Opportunity Solution Tree in tools like Miro, Notion, or FigJam.

Recommended Books on Product Discovery

The two most important titles every product person should read:

• Inspired by Marty Cagan

• Continuous Discovery Habits by Teresa Torres

Final Thoughts

Product Discovery doesn’t have to be complicated. The key is to stay curious about your users, test ideas early, and collaborate with your team to build solutions people actually want.

In a world where data is the new oil, querying it shouldn't feel like learning a new language.

And yet, that’s what most non-technical users face when trying to interact with databases.

Enter Vanna AI: a powerful, 100% open-source Retrieval-Augmented Generation (RAG) framework that converts natural language into accurate SQL queries—securely, scalably, and seamlessly.

With over 14,000 GitHub stars, Vanna has captured the attention of data teams, product managers, and developers for one simple reason:

It just works.

https://youtu.be/hhWId32mpyI

🚀 What is Vanna AI?

Vanna AI is an open-source Text-to-SQL engine built on RAG (Retrieval-Augmented Generation), specifically designed to handle complex, dynamic datasets like the ones used in modern enterprises.

At its core, Vanna makes databases talk back in plain English. Whether you're working with Snowflake, Redshift, Postgres, or BigQuery, Vanna translates your questions into SQL queries that run directly on your data.

🧠 How Vanna Works in 2 Steps

Using Vanna is surprisingly simple:

1️⃣ Train a RAG model on your database schema, documentation, and query logs (it creates a knowledge base).

2️⃣ Ask your business question in plain English. Vanna generates SQL that can directly run on your DB—or be reviewed by analysts before execution.

And yes, it gets smarter with each question.

🔑 Key Features

FeatureDescription🎯 High AccuracyHandles nested, complex SQL queries with better performance than most LLM-only tools.🧠 Self-LearningContinuously improves by learning from past queries and feedback.🔒 Secure by DesignYour data stays within your environment—Vanna only needs metadata, not actual records.🌐 Database-AgnosticSupports all major SQL databases: Postgres, Snowflake, Redshift, MySQL, SQL Server, and more.💻 Flexible FrontendsIntegrate Vanna with Jupyter Notebooks, Slack bots, Streamlit apps, or even Flask APIs.

⚙️ Under the Hood: Architecture Overview

Vanna combines the best of three layers:

1. LLMs (OpenAI, Anthropic, Gemini, etc.) – For interpreting natural language and generating human-like SQL.

2. Vector DBs (ChromaDB, FAISS, Qdrant, etc.) – To search and retrieve relevant metadata, schemas, or logs.

3. SQL Connectors – For seamless execution or validation on live databases.

The stack is modular—you choose which LLM, which vector store, and which database you want to plug in.

💡 Use Cases

Vanna isn’t just a technical marvel. It’s also practically useful:

• 📊 Business Intelligence: Non-technical users can query without writing SQL.

• 🔍 Exploratory Analysis: Analysts can speed up data discovery.

• 👨‍🏫 Education: Helps learners understand the SQL equivalent of business questions.

• 🛠 Internal Tools: Integrate into Slack, dashboards, or internal portals for real-time data Q&A.

🔧 Installation & Example

Want to try it? Just:

pip install vanna

Note: Depending on the LLM and vector store used, additional packages may be required.

Then Basic Setup (OpenAI + ChromaDB Example)

from vanna.openai.openai_chat import OpenAI_Chat
from vanna.chromadb.chromadb_vector import ChromaDB_VectorStore
class MyVanna(ChromaDB_VectorStore, OpenAI_Chat):
    def __init__(self, config=None):
        ChromaDB_VectorStore.__init__(self, config=config)
        OpenAI_Chat.__init__(self, config=config)
vn = MyVanna(config={'api_key': 'your-openai-key', 'model': 'gpt-4'})
vn.train(schema='your_schema', logs='your_query_logs')
vn.ask("What are the top 5 products by revenue?")

🎓 Training Vanna AI

You can improve Vanna's output quality by training it on your:

✍️ DDL Statements

vn.train(ddl="""
    CREATE TABLE IF NOT EXISTS my_table (
        id INT PRIMARY KEY,
        name VARCHAR(100),
        age INT
    )
""")

📚 Business Documentation

vn.train(documentation="Our business defines XYZ as ...")

📈 SQL Query Examples

vn.train(sql="SELECT name, age FROM my_table WHERE name = 'John Doe'")		

💬 Ask Questions

vn.ask("What are the top 10 customers by sales?")	

Returns:

SELECT c.c_name AS customer_name,
       SUM(l.l_extendedprice * (1 - l.l_discount)) AS total_sales
FROM   lineitem l
JOIN   orders o ON l.l_orderkey = o.o_orderkey
JOIN   customer c ON o.o_custkey = c.c_custkey
GROUP BY customer_name
ORDER BY total_sales DESC
LIMIT 10;

And if connected to a DB, you also get a result table and a Plotly chart.

RAG vs Fine-Tuning

RAG ApproachFine-TuningEasily replace LLMsHardcoded to one modelCheap to runExpensive to train & useEasily editable knowledge baseHard to remove outdated training data

Why Vanna AI?

• ✅ Accuracy-first: Improves with more training data

• 🔒 Data secure: SQL execution & training remain local

• 🥇 Customizable: Choose your own stack (LLM + Vector Store)

• 🌟 Ready for production: Jupyter for devs, Streamlit or Slack for users

Interfaces You Can Build With

• Jupyter notebooks (great for data analysts)

• Streamlit apps (dashboard-ready)

• Slack bots (team-friendly)

• Flask APIs (for integration into internal tools)

• Chainlit (chat-style data assistants)

What It Doesn’t Do (Yet)

• Data modification (INSERT/UPDATE) is not the focus.

• Not optimized for non-SQL data sources (e.g., NoSQL).

• The full install can be bulky (~400MB)—but it’s modular.

Final Thoughts

Tools like Vanna AI represent a shift in how we think about querying data. Instead of expecting everyone to learn SQL, Vanna brings the query layer to where the user is—in plain language.

And because it’s open-source and modular, you’re in full control. No vendor lock-ins. No cloud-only limitations. Just powerful, customizable NLP-to-SQL magic.

📚 Learn More & Contribute

Explore the full documentation and source code on the official GitHub repo.

Have a use case in mind? Fork it. Tweak it. Or just start asking questions—and let your data speak.

Every product team loves a well-crafted persona: a neat profile that neatly summarizes who they’re designing for. But all too often, that persona is less “real user” and more “wishful thinking.” We scribble down demographic details—age, family status, job title—and voilà: our roadmap is set, our designs are approved, and our marketing copy writes itself. Yet these static snapshots can mask the messy, unpredictable realities of people’s lives. When those realities emerge, it can feel like the rug has been pulled out from underneath your roadmap.

📌 Case Study: The “Ideal” Indian Teacher Persona

A few years ago, I consulted in a major EdTech in India. Their target user was crystal clear on paper:

Name: “Priya”
Age: 35
Family: Married with two kids, ages 4 and 7
Job: Government school teacher in a Tier‑3 town
Lifestyle: Supportive husband and in‑laws help with childcare
Motivation: Eager to adopt new classroom tools

To anyone glancing at these bullet points, Priya seemed like a dream user: mature enough to understand pedagogy, curious enough to embrace innovation, and well‑supported enough to find the time. It felt like we had struck gold—or so we thought.

🔍 Uncovering the Reality: Primary Research Insights

When we finally sat down with real teachers, everything about our Priya persona began to unravel.

AssumptionRealityAge: Mid‑30sMost women teaching with kids aged 4 and 7 are actually in their late 20s—early marriage and early careers are common.Supportive Home LifeInstead of help, many teachers return home to a “second shift” of cooking, cleaning, and childcare—no in‑laws in sight.Job by ChoiceTeaching is often a necessity—a steady salary that fits within societal expectations, not a passion project.Tech CuriosityComplex tools threaten classroom authority. If students master a tool faster, the teacher’s role feels undermined.Leisurely Learning TimeThere are no 30‑minute coffee breaks. Any “learning” has to happen in stolen 5‑minute windows between classes and chores.

Key Takeaways

1. Misaligned Motivations: Teaching isn’t “fun.” It’s a job that fits cultural and financial needs.

2. Scarce Time & Energy: At best, a five‑minute gap to explore a new feature—never a half‑hour deep dive.

3. Authority Preservation: Tools must reinforce the teacher as the expert, not expose them to potential embarrassment.

💥 What It Broke

The original persona had shaped everything:

• ✍️ Copy assumed curiosity.

• 🧭 Onboarding assumed available time.

• 🔧 Feature complexity assumed confidence.

Once we saw the reality, we had to reframe everything:

• Simplify interfaces—no learning curve, no ambiguity.

• Design for low attention spans—5-minute windows, not 30-minute explorations.

• Reinforce teacher authority—tools had to feel like enhancements, not risks.

What we had built made sense for the imagined persona. But for the real one, it added stress.

🧭 The Real Lesson

Most teams don’t ignore users—they just rush to define them. And once a persona is blessed, it becomes gospel.

Here’s the blind spot:

⚠️ Teams guess their way into personas. Then build real products on imaginary users.

⚠️ They focus on surface traits—age, job title, family—without understanding what life actually feels like.

⚠️ And once a persona is socialized, it becomes sticky. Even when the truth emerges, changing course feels hard.

👉 A persona is not a user. It’s a hypothesis.

👉 A real persona includes constraints, context, and coping strategies.

👉 The job of the product team is to stay curious—especially after the first draft.

Wrapping It Up

It’s easy to fall in love with a persona that sounds good in a slide deck. But product decisions don’t happen in the abstract—they hit the ground in real people’s lives.

When you skip context, you don’t just miss nuance—you risk building entirely for the wrong person.

So treat personas as living documents. Pressure-test them. Revisit them often. Because the cost of getting them wrong shows up where it hurts most: in adoption, trust, and usability.

Imagine having a voice assistant that doesn't just respond from predefined answers but actually retrieves relevant context from your own local documents and responds using your own cloned voice.

Welcome to the era of real-time voice-based Retrieval-Augmented Generation (RAG) agents!

In this post, I’ll walk you through:

• How I built a real-time voice-based RAG agent

• How I integrated AssemblyAI for transcription

• How the agent uses LiveKit for voice streaming

• How it responds in my cloned voice using Cartesia AI

• Practical real-life use cases that make this more than a fun demo

⚙️ Architecture Overview

Here’s what’s happening under the hood:

Technologies Used:

• 🧠 LLM: Google’s Gemma 3 via LlamaIndex

• 📄 Indexing: Local PDFs indexed with LlamaIndex

• 🗣️ STT: AssemblyAI

• 🧏‍♂️ TTS: Cartesia AI

• 🎤 VAD: Silero

• 📡 Orchestration: LiveKit agents playground

🛠️ Building the Agent

For those interested in the architecture, you can check out the open-source code here:

GitHub

Or follow the instructions below:

1. Set up your environment

pip install -r requirements.txt						

Create a .env file with:

OPENAI_API_KEY=your_openai_api_key
CARTESIA_API_KEY=your_cartesia_api_key
LIVEKIT_URL=your_livekit_url
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret
ASSEMBLYAI_API_KEY=your_assemblyai_api_key			

2. Index your documents

Place your PDFs in a docs/ folder and run:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
reader = SimpleDirectoryReader("docs")
docs = reader.load_data()
index = VectorStoreIndex.from_documents(docs)
index.storage_context.persist(persist_dir="chat-engine-storage")

3. Configure your agent (Python)

Use the VoicePipelineAgent from LiveKit. Key configurations:

agent = VoicePipelineAgent(
    vad=silero.VAD.load(),
    stt=assemblyai.STT(),
    llm=llama_index.LLM(chat_engine=chat_engine),
    tts=cartesia.TTS(
        model="sonic-2",
        voice="your-cloned-voice-id",
    ),
    chat_ctx=chat_context,
)		

4. Run the agent

python voice_agent.py start		

Now head to agents-playground.livekit.io, and click Connect. Speak your question out loud—something relevant to the documents you indexed in Step 2.

🧠 The agent will detect your voice, transcribe it using AssemblyAI, fetch context from your documents, generate a response using Gemma 3, and speak it back using Cartesia's voice synthesis engine.

5: Clone Your Voice with Cartesia

By default, your agent uses a robotic voice provided by Cartesia. But you can replace that with your own cloned voice.

1. Visit Cartesia.ai

2. Click on Instant Voice Clone

3. Provide a name and a short 5-second audio sample (e.g., "Hey, this is Aditya and this is my voice for cloning")

4. Cartesia will return a voice_id

5. Replace the default voice ID in your .env file:

CARTESIA_VOICE_ID=your_cloned_voice_id			

6. Restart your agent to use your own voice!

Below is the full Python code I used to build this agent:

import logging
import os
from dotenv import load_dotenv
from livekit.agents import JobContext, JobProcess, WorkerOptions, cli
from livekit.agents.job import AutoSubscribe
from livekit.agents.llm import (
    ChatContext,
)
from livekit.agents.pipeline import VoicePipelineAgent
from livekit.plugins import cartesia, silero, llama_index, assemblyai
load_dotenv()
logger = logging.getLogger("voice-assistant")
from llama_index.llms.ollama import Ollama
from llama_index.core import (
    SimpleDirectoryReader,
    StorageContext,
    VectorStoreIndex,
    load_index_from_storage,
    Settings
)
from llama_index.core.chat_engine.types import ChatMode
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
load_dotenv()
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
llm=Ollama(model="gemma3", request_timeout=120.0)
Settings.llm = llm
Settings.embed_model = embed_model
# check if storage already exists
PERSIST_DIR = "./chat-engine-storage"
if not os.path.exists(PERSIST_DIR):
    # load the documents and create the index
    documents = SimpleDirectoryReader("docs").load_data()
    index = VectorStoreIndex.from_documents(documents)
    # store it for later
    index.storage_context.persist(persist_dir=PERSIST_DIR)
else:
    # load the existing index
    storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
    index = load_index_from_storage(storage_context)
def prewarm(proc: JobProcess):
    proc.userdata["vad"] = silero.VAD.load()
async def entrypoint(ctx: JobContext):
    chat_context = ChatContext().append(
        role="system",
        text=(
            "You are a funny, witty assistant."
            "Respond with short and concise answers. Avoid using unpronouncable punctuation or emojis."
        ),
    )
    chat_engine = index.as_chat_engine(chat_mode=ChatMode.CONTEXT)
    logger.info(f"Connecting to room {ctx.room.name}")
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
    participant = await ctx.wait_for_participant()
    logger.info(f"Starting voice assistant for participant {participant.identity}")
    stt_impl = assemblyai.STT()
    agent = VoicePipelineAgent(
        vad=ctx.proc.userdata["vad"],
        stt=stt_impl,
        llm=llama_index.LLM(chat_engine=chat_engine),
        tts=cartesia.TTS(
            model="sonic-2",
            voice="794f9389-aac1-45b6-b726-9d9369183238",
        ),
        chat_ctx=chat_context,
    )
    agent.start(ctx.room, participant)
    await agent.say(
        "Hey there! How can I help you today?",
        allow_interruptions=True,
    )
if __name__ == "__main__":
    print("Starting voice agent...")
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
        ),
    )
				
			

🧠 Real-Life Use Cases

Here’s where this gets seriously powerful:

1. Education & Research

Ask research papers questions by simply speaking. No UI. No ChatGPT. Just voice.

2. Enterprise Knowledge Assistants

Replace dumb IVRs. Let users talk to internal wikis, HR docs, SOPs.

3. Elderly or Visually Impaired Users

Make documentation searchable and explorable without screens.

4. Developer Agents

Let engineers talk to API docs, CLI tools, changelogs—all via voice.

5. Personalized Voice Agents

Build your own Jarvis-style assistant trained on your workflows.

Stay tuned for more experiments in real-time AI agents 🚀

Optical Character Recognition (OCR) is an essential tool for extracting text from images, making it easier to digitize documents, analyze handwritten notes, and process structured information. With advancements in AI, OCR has become more accessible and efficient, allowing developers and product managers to integrate it into their workflows seamlessly.

In this guide, we’ll walk you through building your own locally running OCR app using Google’s Gemma-3 Vision model, Ollama, and Streamlit. This app allows users to upload an image and extract structured text in Markdown format, ensuring clarity and organization.

What is Ollama, Gemma-3, and Streamlit?

Before we dive into the setup, let’s understand the key components of our application:

• Ollama: A lightweight tool that allows you to run large language models locally on your machine. It helps deploy models efficiently without requiring cloud-based services.

• Gemma-3 Vision Model: A powerful multimodal AI model developed by Google that can process both images and text, making it ideal for OCR tasks.

• Streamlit: A Python-based framework that makes it easy to create interactive web applications for machine learning models and data visualization.

With these three tools, we can build a fully functional OCR app that runs entirely on your local machine.

Step-by-Step Guide to Set Up and Run the OCR App

Github

1. Install Ollama

To run Gemma-3 locally, we first need to install Ollama.

On Linux:

curl -fsSL https://ollama.com/install.sh | sh

On Mac:

brew install ollama

On Windows (using WSL 2):

wsl --install
curl -fsSL https://ollama.com/install.sh | sh			

After installation, verify that Ollama is installed by running:

ollama --version			

2. Download the Gemma-3 Vision Model

Once Ollama is set up, download and load the Gemma-3 Vision model:

ollama run gemma3:12b

This command ensures that the model is available on your local machine for inference.

3. Install Python and Required Dependencies

Ensure you have Python 3.11 or later installed. Then, install the necessary Python packages:

pip install streamlit ollama pillow	

4. Create the Streamlit App

Now, let’s write the Python script that powers our OCR application. Save the following code as app.py:

import streamlit as st
import ollama
from PIL import Image
import io
import base64
# Page configuration
st.set_page_config(
    page_title="Gemma-3 OCR",
    page_icon="🔎",
    layout="wide",
    initial_sidebar_state="expanded"
)
# Title and description in main area
st.markdown("""
    # <img src="data:image/png;base64,{}" width="50" style="vertical-align: -12px;"> Gemma-3 OCR
""".format(base64.b64encode(open("./assets/gemma3.png", "rb").read()).decode()), unsafe_allow_html=True)
# Add clear button to top right
col1, col2 = st.columns([6,1])
with col2:
    if st.button("Clear 🗑️"):
        if 'ocr_result' in st.session_state:
            del st.session_state['ocr_result']
        st.rerun()
st.markdown('<p style="margin-top: -20px;">Extract structured text from images using Gemma-3 Vision!</p>', unsafe_allow_html=True)
st.markdown("---")
# Move upload controls to sidebar
with st.sidebar:
    st.header("Upload Image")
    uploaded_file = st.file_uploader("Choose an image...", type=['png', 'jpg', 'jpeg'])
    if uploaded_file is not None:
        # Display the uploaded image
        image = Image.open(uploaded_file)
        st.image(image, caption="Uploaded Image")
        if st.button("Extract Text 🔍", type="primary"):
            with st.spinner("Processing image..."):
                try:
                    response = ollama.chat(
                        model='gemma3:12b',
                        messages=[{
                            'role': 'user',
                            'content': """Analyze the text in the provided image. Extract all readable content
                                        and present it in a structured Markdown format that is clear, concise, 
                                        and well-organized. Ensure proper formatting (e.g., headings, lists, or
                                        code blocks) as necessary to represent the content effectively.""",
                            'images': [uploaded_file.getvalue()]
                        }]
                    )
                    st.session_state['ocr_result'] = response.message.content
                except Exception as e:
                    st.error(f"Error processing image: {str(e)}")
# Main content area for results
if 'ocr_result' in st.session_state:
    st.markdown(st.session_state['ocr_result'])
else:
    st.info("Upload an image and click 'Extract Text' to see the results here.")
# Footer
st.markdown("---")
st.markdown("Made with ❤️ using Gemma-3 Vision Model | [Adityeah](https://adityeah.in/)")
				
			

5. Run the Streamlit App

Once the script is ready, launch the Streamlit app by running:

streamlit run app.py

Your OCR app will be accessible in your browser, allowing you to upload an image and extract structured text.

Use Cases of OCR Across Domains

Education

1. Automated Exam Grading: Teachers can upload images of students' answer sheets, and the model can compare student responses with predefined correct answers to automate grading and feedback generation.

2. Homework Evaluation: Students can submit handwritten homework, which the model can analyze for correctness and provide instant feedback.

3. Doubt Resolution: Students can upload handwritten queries, and the model can extract and structure the content for quick reference and resolution.

4. Digitizing Study Materials: OCR can convert handwritten notes, books, and lecture slides into digital formats for better accessibility.

Finance

1. Automated Invoice Processing: OCR extracts and processes financial data from invoices and receipts for accounting automation.

2. Check and Bank Statement Scanning: Quickly digitize paper checks and bank statements to track transactions more efficiently.

3. Fraud Detection: OCR can verify documents for discrepancies in financial reports and detect potential fraud.

Healthcare

1. Medical Record Digitization: Convert handwritten doctor notes and prescriptions into electronic health records (EHRs) for better accessibility.

2. Insurance Claims Processing: Extract data from claim forms and automate insurance approvals.

3. Patient Form Processing: Reduce paperwork by digitizing patient intake forms and medical histories.

Retail & Logistics

1. Barcode and Label Scanning: Automate inventory tracking by extracting data from labels and barcodes.

2. Automated Receipt Processing: Digitize and categorize receipts for expense tracking and reimbursement.

3. Order Management: OCR extracts shipping and order details from printed documents to streamline logistics operations.

With OCR technology becoming more powerful and accessible, businesses across industries can leverage it to automate workflows, enhance efficiency, and improve data accuracy.

One of the most common insecurities for product managers (PMs) is their technical skills. While you don’t need to be an engineer, having a solid understanding of how software works can help you communicate effectively with your team, manage risks, and make better product decisions. Let’s break down the technical jargon and concepts you need to know to feel more confident in 2025 and beyond.

How Internet Software Works: The Core Components

All internet-based software products follow the same foundational architecture. They consist of three main components:

1. Client: This is what your users interact with directly. Examples include websites, mobile apps, smart devices like Tesla cars, or even smart fridges.

2. Server: This is the brain of the operation. It processes requests from the client, fetches or stores data, and performs actions like syncing your Google Calendar.

3. Database: This is where all the data is permanently stored. Think of it as the filing cabinet for your application.

Each of these components can be broken down into smaller, more specialized parts. Building scalable, well-organized software requires careful planning and execution. Let’s dive deeper into each of these components.

Frontend: What Users See and Interact With

The frontend is the part of the application that users interact with directly. It’s the code that runs on the user’s device, whether it’s a browser, a mobile app, or a smart device. Frontend development involves creating intuitive, responsive, and visually appealing interfaces.

Key Technologies in Frontend Development

• HTML (HyperText Markup Language): This defines the structure of a webpage. It outlines where elements like headings, paragraphs, and buttons appear.

• CSS (Cascading Style Sheets): This controls the styling of the page, including fonts, colors, spacing, and layouts.

• JavaScript (JS): This adds interactivity to the page. For example, it allows animations to play, forms to be submitted, or new data to be loaded without refreshing the page.

Frontend Frameworks

As web applications have grown more complex, developers have turned to frameworks to streamline their work. These frameworks provide pre-built tools and components, making it easier to organize code and build scalable applications. Popular frontend frameworks include:

• React: Developed by Facebook, React is widely used for building dynamic user interfaces.

• Vue: Known for its simplicity and flexibility, Vue is a great choice for smaller projects or teams.

• Angular: A robust framework maintained by Google, Angular is ideal for large-scale enterprise applications.

How Frontend and Backend Communicate

The frontend communicates with the backend using HTTP requests. These requests are sent to specific endpoints (like a mailbox) on the server. Common types of HTTP requests include:

• GET: Retrieve information (e.g., loading a user’s profile).

• POST: Submit information (e.g., creating a new account).

• PATCH/PUT: Update information (e.g., changing a user’s password).

• DELETE: Remove information (e.g., deleting a post).

Backend: The Engine Behind the Scenes

The backend is where the magic happens. It’s the part of the application that users don’t see but is critical for processing data, handling business logic, and ensuring everything runs smoothly. Backend code is not distributed to users—it runs on servers hosted in the cloud.

Cloud Hosting Providers

Most modern backend systems are hosted on cloud platforms like:

• Amazon Web Services (AWS): The most widely used cloud provider, offering services like EC2 (servers), S3 (file storage), and Lambda (serverless functions).

• Google Cloud Platform (GCP): Known for its data analytics and machine learning capabilities.

• Microsoft Azure: A popular choice for enterprises, especially those already using Microsoft products.

These platforms allow you to rent computing resources like servers, databases, and storage, which can be scaled up or down based on demand.

Key Backend Components

• API Gateways: Act as the front door to your backend, routing incoming requests to the appropriate services.

• Caches: Temporary storage for frequently accessed data, speeding up response times.

• Queues: Help manage high volumes of requests by storing them until they can be processed.

• Content Delivery Networks (CDNs): Store static files like images and videos in multiple locations worldwide to reduce load times for users.

Scaling: Handling Growth Without Breaking

As your product grows, so does the demand on your backend. Scaling is the process of adding resources to handle increased traffic and ensure your application remains fast and reliable.

Types of Scaling

1. Horizontal Scaling: Adding more machines to distribute the load. For example, going from 1 server to 10 servers.

2. Vertical Scaling: Upgrading existing machines with better hardware, such as more memory or faster processors.

Challenges of Scaling

Scaling isn’t just about adding more resources—it’s about ensuring the entire system can handle the increased load. Common challenges include:

• Bottlenecks: One component (e.g., the database) may become a bottleneck if it can’t handle the increased traffic.

• Cost: Scaling can become expensive, so it’s important to optimize resources.

• Complexity: As you add more components, the system becomes harder to manage.

Tools for Scaling

• Pub/Sub: A messaging system where one server publishes a message, and others subscribe to it.

• Kubernetes: A tool for automating the deployment and management of containers (packaged applications).

• Batching: Grouping multiple requests together to improve efficiency.

• Rate Limiting: Controlling the number of requests a user or service can make in a given time period.

Common Backend Terms Every PM Should Know

• Pubsub: A queue. A message is sent in to the queue (published) and one or more servers can receive it (subscribe)

• EC2: A product offered by AWS to run a server (Elastic Compute Capacity)

• S3: A product offered by AWS to store files (Simple Storage Service)

• SQS: A product offered by AWS to run a queue (Simple Queue Service)

• SES: A product offered by AWS to send emails (Simple Email Service)

• Lambda: A product offered by AWS to rent a server for a very short period of time

• Containers: Document everything you need to run your application in one file

• Kubernetes: Deploy a container to a server and manage all your servers automatically

• Batching: Grouping requests together instead of sending them one at a time

• Rate limit: The maximum number of requests that can be processed in some duration

• Microservice: A set of resources (server, database, queue) that are built to do a specific task, like handling billing, notifications, recommendations, etc.

Why This Matters for Product Managers

Understanding these technical concepts isn’t about replacing your engineering team—it’s about empowering yourself to:

• Communicate Effectively: Speak the same language as your engineers and align on goals.

• Manage Risk: Identify potential technical challenges early in the product development process.

• Make Informed Decisions: Evaluate trade-offs between different technical approaches.

The TL;DR

A relational database is sort of like Excel, but for developers: it’s how applications store and analyze data.  

• Relational databases like MySQL and PostgreSQL are the default software for storing data

• Relational DBs are all about schemas: rules for how data is stored and organized

• Inserting data into relational DBs follows the ACID specification

• SQL is a type of programming language that lets you pull and analyze the data in your relational DB

Relational DBs power at least part of almost every single application that you use in your day to day. So read this.

What’s a relational database?

→ First: what’s a database in general

A database is just a place to store data; even Excel is technically a database. But when developers talk about databases, they’re usually referring to specialized software that’s used to store data from applications. When you sign up for Facebook, a new entry gets added to Facebook’s users database with a bunch of information about you; every time you sign in after that, that data gets pulled and used (this is part of what a backend is).

There are thousands of different databases out there, all optimized for different types of applications, some free and some paid. Some popular ones you might have heard of: MySQL, PostgreSQL, and MongoDB. 

🚨 Confusion Alert 🚨

People use “database” to refer to a lot of different things: most of the time it’s our topic (programmatically accessed data stores), but you might also hear about “my database of top CEOs” or something like that, which is probably a spreadsheet. Just listen for the context and you should be fine. 

🚨 Confusion Alert 🚨

→ Second: what’s a relational database

Relational databases are the most popular kind of DBs, and they’re pretty simple: they add rigidity and rules around how data is stored. To understand what that means, let’s imagine we’re moving across the country (yes I’m still crying, no I don’t want to talk about it). You’ve got a ton of stuff in your house: clothes, furniture, random keychains that you can’t let go of, and kitchenware, among other things. How do you store / pack all of that?

• You could throw everything in random boxes, like back in college 

or

• You could organize things by size or category, and pack and label specific boxes

There’s a clear tradeoff here between speed and efficiency. Packing properly takes a lot more time, but allows you to make better use of space and more easily find your stuff later. It turns out that databases have the exactly same tradeoff, and relational DBs are like packing properly instead of throwing stuff in. 

In relational databases, you specify everything about your data up front: 

• What type it is (text, number, yes/no, etc.)

• How it’s organized (different tables, how those tables are related to each other, etc.)

• Names for everything (columns)

All of this together is called a schema, and it’s the basis of relational DBs. But why are they called relational?

Relations, take 2

A relational database is called what it is because of the relationships it models between your data.

In a relational DB, data is organized into tables with rows and columns: you might have a table for data about your app’s users, as well as a table for your app’s orders. Those tables are related, because each order has a user that made that order. Big applications can have hundreds or thousands of those tables, and you need to define the relations between them for it to work well. 

In this diagram (and, I’m guessing, in Amazon’s internal systems) there’s a users table and an orders table: the relation between those tables is that each order has one user who created it. 

There’s a good reason why things are designed this way instead of throwing all of your data into one giant table: it makes inserting and pulling data way easier and more efficient. In the early days of the internet, when this technology was just starting out, it was incredibly expensive to store and retrieve data. So people spent a ton of time on optimizing it: making sure the same data is never stored twice (this is called normalization), finding the most efficient way to store different data types, and all of these low level things that are beyond my pay grade. 

Transactions and SQL

Just in case there weren’t enough acronyms yet, I gotchu.

→ ACID transactions

Modern software applications need to be fast and make few or no mistakes. Facebook can’t mess up your login, or mistake you for another user: that’s why relational DBs have special features around transactions. A transaction is just an interaction with the database, either inserting data into it or querying and pulling data from it. Transactions can go wrong all the time for a ton of different reasons, but there’s a body of standards called ACID (broooo) that helps avoid them. 

The specifics of ACID aren’t too important. Just keep in mind that if you see a phrase like “ACID compliant transactions” it’s referring to these standards.

→ SQL

Pulling data from a database is called querying, and there’s a series of special programming languages that let you do that called Structured Query Language (SQL). There are all different types and flavors of SQL (depending on which relational DB you’re using), but they’re all pretty similar. A typical SQL statement might look like this:

				
					SELECT 
    first_name,
    last_name,
    address,
    last_order_date
FROM users
WHERE signup_date > '2019-01-01'
				
			

SQL is pretty human readable – even if you don’t know it, you can guess what we’re doing here – and it’s a really useful thing for everyone to know, even if you’re not en engineer. 

☕️ Dev Culture ☕️

Software developers need to know basic SQL for building their apps, but the biggest SQL experts are usually Data Scientists and Analysts: their entire days (I was one!) are spent querying and manipulating data from databases. For that and a few other reasons, some developers treat SQL as a second-tier, not “real” programming language.

☕️ Dev Culture ☕️

Databases are the unsung heroes of modern technology. They power everything from your favorite social media apps to the analytics dashboards that businesses rely on for decision-making. But with over 300+ databases available, each designed for specific use cases, choosing the right one can feel overwhelming. Whether you're a  seasoned developer or an aspiring PM, understanding databases is crucial for building efficient, scalable, and reliable systems.

In this guide, we’ll explore databases in depth, using relatable metaphors, real-world examples, and clear explanations to help you understand how they work, why they matter, and how to choose the right one for your needs.

The Flour Metaphor: Databases as Ingredients

Let’s start with a metaphor: baking bread.

When baking, the type of flour you use determines the texture, flavor, and structure of your bread. High-protein flour (like bread flour) creates a chewy, dense loaf, perfect for pizza or bagels. Low-protein flour (like cake flour) makes light, flaky pastries. All-purpose flour works for everything, but it’s not ideal for specialized recipes.

Databases are like flour. Each one is designed for specific use cases. You can use any database for anything, but choosing the right one ensures the best possible bread outcome. To understand which database to use, ask two key questions:

1. What does the data look like?

2. What do you need to do with it?

Let’s break these down.

1. What Does the Data Look Like?

Databases are optimized for specific types of data. While you can store any piece of data in any database, the way data is inserted and retrieved matters most. Postgres is relational, but supports big blobs of text. Nothing is stopping you from storing your user data in Elastic. Nevertheless, many databases are designed with specific types or shapes of data in mind.

Here’s a closer look at how data shape influences database choice:

Structured Data

• What it is: Data with a clear, predefined schema (e.g., tables with rows and columns).

• Example: User profiles, orders, or inventory.

• Best databases: Relational databases like Postgres, MySQL, or MSSQL.

Unstructured or Semi-Structured Data

• What it is: Data without a fixed schema (e.g., JSON documents, logs, or social media posts).

• Example: Tweets, sensor data, or product reviews.

• Best databases: NoSQL databases like MongoDB, Cassandra, or Elasticsearch.

Interconnected Data

• What it is: Data with complex relationships (e.g., social networks, recommendation engines).

• Example: Friend connections on Facebook or product recommendations on Amazon.

• Best databases: Graph databases like Neo4J.

Time-Series Data

• What it is: Data indexed by time (e.g., stock prices, sensor readings).

• Example: Hourly temperature data or daily financial transactions.

• Best databases: Time-series databases like InfluxDB or TimescaleDB.

2. What Do You Need to Do with the Data?

Different types of data – and even similar types of data for different use cases – get used in different ways. How you use the data determines the database you need. Here are some common use cases and the databases that excel in them:

OLTP (Online Transactional Processing)

These are the databases that you use for the core data in your application, think your user data. Data is being read and inserted constantly. Queries are usually small and quick. Data integrity is key.

• What it is: Real-time, transactional data processing (e.g., user actions, payments).

• Example: Adding items to a shopping cart or updating a user profile.

• Best databases: Relational databases like Postgres or MySQL.

OLAP (Online Analytical Processing)

These are the databases that you use for analytics and data science, think Snowflake etc. Data is being read and inserted sporadically. Queries are usually large and complex. 

• What it is: Complex queries on large datasets for analytics or business intelligence.

• Example: Calculating monthly revenue or analyzing user behavior.

• Best databases: Data warehouses like Snowflake, BigQuery, or Redshift.

Operational Use Cases

• What it is: Logs, monitoring, caching, or internal processes.

• Example: Storing server logs or managing user sessions.

• Best databases: Elasticsearch for logs, Redis for caching, or Prometheus for monitoring.

The fact that users of the database need to do different things with the data inside of it necessitates storing said data differently. 

OLTP vs. OLAP is one example, but there are many different ways to break down those use cases. Some databases have special UIs on top for analysis (like Kibana and Elastic). Others are built for massive scale from the start. Some are made to handle many different situations decently well. No matter what database you’re dealing with, thinking about what kind of data it stores and how you need to use that data is the key to understanding each.

3 major categories of databases and what they do

Crude as it may be, I find it useful to think of 3 major categories of databases. Some databases can overlap and cross categories. For example, there are NoSQL databases built for powering your app, but there are also NoSQL databases meant for in-memory stuff (e.g. Redis is technically NoSQL). Here are the major ones:

1. Databases that power a user-facing app – production databases that store the data you need for your app to run.

2. Databases that power analytics – databases for analysis, machine learning, and anything a data team does.

3. Databases that power operations – databases for monitoring, logs, security, and any internal processes that enable the above.

Let’s run through each category and cover a few examples for each.

1. Databases That Power User-Facing Apps

Databases that power a user-facing app – often referred to as production databases or production data stores – are where developers store and query whatever data their app needs to run. Any “data” you’re seeing on your screen sits here.

• If you’re Twitter: tweets, user profiles, trending topics, DMs

• If you’re Gmail: emails, settings, spam filters

• If you’re Amazon: orders, users, credit cards

Production databases are built to support small and quick queries. The most important thing is data integrity and reliability – you don’t want to lose an order or a credit card.

A. User-facing DBs / Relational Databases

1. Relational databases: are the database OGs. They’ve been around for pretty much as long as databases have been around, and are the default choice today for storing the basic data your application needs to run. Data is stored in a very structured format with rigid definitions of tables, columns, and how tables relate to each other; that way, queries can run quickly and reliably.

2. Postgres is an open source relational database initially released back in 1997. It’s one of the standard “boilerplate” choices for teams getting started on a new app. You can run Postgres yourself on a server, or pay someone like Amazon to run it for you.

3. MySQL is also an open source relational database, initially released in 1995. It’s the other database most teams reach for, although since Oracle bought Sun (and ergo MySQL), there are legal ramifications to modifying the MySQL code for commercial use. In terms of MySQL vs. Postgres, the short story is that they’re very, very similar but have a few important differences with how they handle data types, storage engines, and a few other things.

4. MSSQL is Microsoft’s proprietary SQL database. It’s popular among very large companies but not for smaller startups.

5. Oracle was the first commercial relational database ever. Today it’s got the reputation of being the slowest moving company in California, but nevertheless powers the apps of tons and tons of Fortune 50 companies. Like MSSQL, this is a database for massive organizations and not startups.

6. MariaDB is another open source relational database, from the original creators of MySQL. They (in a somewhat cheeky fashion) seem to be pretty pissed about Oracle limiting contributions to MySQL, so they focus on how open the community is and how the software will never require a license.

7. SQLite is a software library that lets you run a small database inside of your application. It’s used in one way or another in tons and tons of apps, usually as an intermediate database as opposed to a core production store. The website says it’s on every single Android, iPhone, etc.

B. User-facing DBs / NoSQL Databases

1. NoSQL: databases remove the structure and rigidity from relational databases, and let you just dump data in there and worry about how it looks later. There are NoSQL databases for all different categories of database (analytics, operational) but these here are for production, user-facing use.

2. MongoDB popularized the NoSQL moniker and is the default choice if you’re looking for a NoSQL database to power your app. Their cloud product, Atlas, is the AWS RDS equivalent for basic NoSQL. 

3. Cassandra is a NoSQL database built for really big companies who need to store lots of data and retrieve it fast. Unlike MongoDB, which is built as a document database, Cassandra is oriented more around a columnar setup, which means data is stored in entire columns (like Snowflake, actually). Using Cassandra feels a lot more like using a relational database. Thank you to Neo Kim from the System Design Newsletter for pointing out that Cassandra is actually technically not a columnar data store, but instead implements the concept of a wide column family, which is slightly different.

4. DynamoDB is AWS’s proprietary NoSQL database. 

5. Firebase is a series of tools (acquired by Google back in the day) for building apps, targeted at smaller teams and focusing on simplicity. Their database is called Firestore and it’s a really popular option for a quick, just-get-me-started kind of DB. 

C. User-facing DBs / Graph Databases

Every company’s data is different, and for some it makes the most sense to model things as a sort of graph of interconnected nodes. Facebook famously runs their social graph on an in house graph database called Tao. The use cases for graph DBs are mostly user-facing, but they’re sometimes useful for analytical purposes too.

1. Neo4J is the most popular graph database. It’s open source but if you’re going to scale it up, you’ll need to talk to them about their enterprise license. Data in Neo4J is represented as nodes, and nodes can be connected to each other in different ways. Cyper is the name of their SQL-esque language for querying the graph.

2. Databases that power analytics

What a data team needs from their database is very different from developers working on an app. For data science and analytics work, the data you store is often redundant, there’s tons of it, and your queries join data from multiple tables at once. Data usually gets queried by someone sitting at a computer doing research, or a system building a pipeline, and gets inserted at regular intervals (twice a day, something like that). 

A. Analytical DBs / Data Warehouses

Data warehouses are (usually) relational databases for storing analytical data, like what your users have been doing, revenue by month, things like that. They’re optimized for big, long, multi-table queries. They’re usually relational databases in nature, although the implementation details can vary.

1. Snowflake is a cloud data warehouse for analytics. It’s columnar, which means that data is stored (under the hood) in entire columns instead of rows; this makes large analytical queries faster, so it’s a common choice for how to build analytical DBs.

2. BigQuery is also a cloud data warehouse for analytics. It’s very similar to Snowflake.

3. Redshift is also a cloud data warehouse for analytics. It’s very similar to Snowflake and BigQuery.

4. Clickhouse is an open source (!) data warehouse for analytics. It was originally developed at Yandex, and is getting more popular.

🤔 Why can’t I use Postgres as a data warehouse?

You can, people did for a long time, and some still do. But modern cloud data warehouses are just waaaaay faster for large queries on meaningful quantities of data. This is the thing about databases: you can use anything for anything, but there comes a point where you need a specialized tool for the job.

B. Analytical DBs / Data Lakes

1. Data Lakes are basically giant data safes. You throw whatever you want in there – structured, unstructured, big, small, organized, disorganized, whatever – and then worry about structure when it’s time to get the data out. Contrast that with a data warehouse, which like a relational database, has rigid structure around tables, columns, and data types.

2. S3 is AWS’s object storage solution – usually used for storing things like images and videos for applications – but is also commonly used as a data lake. You can’t “query” S3 in the way that you can a database, so you need to use a layer on top (like AWS’s Lake Formation) to intermediate.

3. Databricks, a company that I wrote about back in the day, sells a product they call “the lakehouse platform.” Under several layers of caked marketing makeup, it’s an open source data lake and storage layer built to resemble a sort of data warehouse; they’re trying to blur the distinction between the two.

Though it’s unlikely you’ll see it in production today, HDFS was a highly popular way to build a data lake before the cloud data warehouse era. It’s powered by Hadoop, one of the original frameworks for performing distributed computations on large groups of data. It was/is notorious for being very, very hard to set up and run.

C. Analytical DBs / GIS Databases

This is a bit of a niche one, but worth mentioning: there’s a class of databases (or in some cases, database extensions) that are purpose built for working with geographical data. GIS stands for Geographic Information System. In terms of the actual data being stored here, it can be anything from points and lines to complex 3D data; structures that don’t map (no pun intended) well to traditional database schemas.

1. PostGIS is a PostgreSQL extension for storing and working with geographical data. It’s a combination of features for storage, analysis (special geographical functions like intersections, measuring distances, etc.), and other miscellaneous stuff like geocoding.

2. Kinetica is an enterprise-focused database for storing geospatial and time series data.

3. Oracle Spatial Database is also an enterprise-focused database for storing geospatial data, from our friends at big red.

Databases that power operations

This is the category you’ll probably encounter least if you’re not an engineer. The 3rd category of databases covers data stores that developers use to power internal operations: monitoring application performance, storing logs and security information, improving application speed, or even intermediate layers between other databases.

Operational DBs / Key Value Stores

While pretty much all of the databases we’ve covered so far store permanent data – saved on a harddrive – there’s a class of databases that only keeps data in memory. They’re meant for ephemeral data that you need to store and use quickly, but you don’t mind if it disappears down the road. These databases are called key value stores (or KV stores) because the way they store data is like a dictionary, where each entry has a key.

1. Redis is the OG key value store. It’s an in memory database that teams use for a bunch of different stuff: building caches, managing authentication sessions, chat and messaging, and any other use cases that prioritize real time, very quick data retrieval. Redis is open source but you can also pay them to host it for you.

2. SingleStore (FKA MemSQL) is an in memory database like Redis, more focused on general purpose kinds of workloads than typical in memory database use cases.

Operational DBs / Time Series Databases

Time series databases are DBs built specifically for storing data on some sort of time frame: daily financial data, second-by-second sensor readings, hour-by-hour health checks on your servers, anything like that. 

1. Timescale is an extension to Postgres, sold as an independent cloud hosted database. It takes everyone’s favorite relational database and adds special time-series specific functionality like automatic partitioning and query optimizations.

2. Prometheus is an open source time series database with built in alerting and visualization.

3. InfluxDB is also an open source time series database.

Operational DBs / Logs and Search Databases

1. Elastic is a NoSQL database built for storing and searching through logs, or very granular records of server performance, API requests, and internal stuff like that. Elasticsearch is commonly used with Kibana, its sister data visualization tool.

2. Solr is also a NoSQL database built for storing and searching through logs.

Product analytics – what the hell is it? Why is Amplitude a public company? What do product managers even do?

This post will dive into how companies analyze what their users are doing in their product. We’ll cover what kinds of questions teams want answers to, how the data gets generated and moved, and what tools are out there to simplify things.

It all starts with decision making

The wonderful world of product analytics starts with knowledge, or at least the eternal yet fleeting desire for it. Startups and large companies alike want visibility into how their users use their digital products. They’re looking to understand things like:

• What are our most popular features?

• Which pages on our website convert the best?

• What areas of our product experience are confusing?

• What are common paths that users take to accomplish something?

Armed with this information, product teams can make educated decisions on what they should be improving and building. These kinds of analytics can also help catch bugs: if the number of users visiting a page drops precipitously one day, chances are something is wrong with that page, and needs fixing (or your analytics are broken 😏). 

Product funnels

Product and growth teams like to use the concept of a funnel to visualize the paths that a user takes through the product. The funnel is wider on top – representing actions that a lot of people take, like visiting a website, and narrower on the bottom, representing actions that fewer people take, like signing up for the product and using it. 

Each stage of the funnel (except the top one) has a conversion rate representing the percentage of people in the previous bucket that made it to the current step. Teams try to benchmark their conversion rates against other products to get a sense of whether they are too low (or hopefully, too high).

🚨 Confusion Alert 🚨

Even though it’s referred to as product analytics, this area has blurry overlap with marketing data. Growth funnels can include data on website visits, and which sources (referrals, organic search, etc.) are driving quality leads. Who is responsible for using this data varies; just don’t be surprised if you see traffic-related information – what you’d normally consider marketing – in this post.

🚨 Confusion Alert 🚨

User research and session recording

Numbers can only go so far in giving product teams a good sense of what their users are actually doing in the product. Sometimes, you just need to see how they navigate around and do their tasks. Traditionally, user research teams would find prospective or existing users, and actually observe them accomplishing common tasks in your app. For example, the user research team at Microsoft might pay users to join a study and ask them to build a basic financial model in Excel; they’ll pay close attention to exactly what the users do and what problems they run into.

More recently, tools like FullStory have popped up, which allow teams to literally record what users are doing in the product during their day to day, making all of this significantly easier. More on these tools later.

How it all works under the hood?

As with all things data, it’s a pain in the ass to arm your company with the data to answer all of the questions product teams have. You need to instrument your product to “record” what users are doing, get that data into your warehouse, transform it into a format that’s useful for analysis, and then build visualizations that give you the information you want. 

This section will walk through how that’s done, and the next will cover newer tools that are making things easier than they were.

Note: most tech companies today are using event-driven architectures like the one we’re about to describe. But there are other, worse ways of doing this.

Instrumenting the product

Step one is to actually create the data. And doing this is fairly simple, albeit custom: you just send what’s called an event any time a user does something in the product. Suppose we’d like to record every time a user clicks on the “save changes” button when editing their profile. We’d fire an event like this:

				
					analytics.track(
  {
    event_name: “save_changes_button_clicked”,
    timestamp: “2022-03-09 23:45:33”,
    context_page: “/profile/edit”,
    user_id: “239hg72eyrhdaadgdg99277hd7ve88d”
  }
)
				
			

The event gets a name, a timestamp, and some metadata: in this case, the context in which it took place (on the edit profile page), and the ID of the user who took the action.

Fun fact, you don’t need to take my word for this. If you’re logged into a SaaS product on your computer right now, open up your developer tools (⌘ + Option + C for chrome) and click on the “network” tab. Click on any button or link in the app, and you should see some events showing up at the bottom. 

🔍 Deeper Look 🔍

The logistics of sending these events gets interesting. Most companies send them client side, e.g. right from your browser. But if you’re using an ad blocker, that blocks any client side JavaScript from hitting external servers, which renders these events useless. For that reason, some companies will set up a proxy server and send these events from their backend; more work, but it avoids the ad blocker problem.

🔍 Deeper Look 🔍

These events get created and fired for pretty much anything you do in the product, and later get assembled into useful insights. But first, the data needs to be stored somewhere.

Getting the data into the warehouse

This is where things start to get complicated. Once the event gets “fired” where does it go? 

These days, the state of the art is to put it straight into your data warehouse. There are a few ways to do that. You can build it yourself – you’d need to build a system that parses the event data and inserts it into Snowflake, BigQuery, etc. via SQL – or, more commonly, use a third party tool like Segment or PostHog to do it for you. 

The end “product” is usually a bunch of tables in your warehouse, one for each type of event. You’ll have a table called `clicked_save_changes`, one called `edited_profile`, and so on and so forth. Sometimes these events will get aggregated into a big events table called, well, `events`. Each one of these tables will have columns for the important data: the time the event happened, which user did it, what page it happened on, etc.

Now we’re getting somewhere!

Transforming the data into something useful

A bunch of raw events aren’t exactly useful for a product manager, even if they can write SQL. How am I supposed to know what my funnel looks like when I’m working with a bunch of random events? The next step in the process is to transform the event data into aggregated, more useful information.

Transforming data today just means writing some SQL that takes these events and filters and aggregates them. For a concrete example, imagine we’re trying to find the most popular “page” in our product, and look at how many users visit each page on a weekly basis. The raw event data looks like this:

But we want the data to look like this:

To do that, we write a bunch of SQL (I will spare you the logistics). State of the art today is to use something like dbt to do this. This is usually the responsibility of the data team, and they will work with business stakeholders like the product team to figure out what these transformations should actually look like.

SQL isn’t necessarily the ideal tool for analyzing event data like this. While we do eventually want to aggregate a lot of this data into rollup tables like the above, sometimes explorations into why users are doing a particular action requires diving into the granular events themselves. Writing custom SQL queries for these over and over again, and analyzing the results tabularly, is a nightmare. In other words, there are still many open questions as to how this type of data should be analyzed most effectively. 

Visualization, BI, and exploration

Once the data is in the format we want it in, we’ll want to get it into a chart of some sort. There are tons of ways to do this, from Metabase to Retool to Hex to homegrown solutions. Even Looker, before it was all but destroyed by acquirer Google, was a tool that teams used for this, among other things. You’ll usually end up with a series of graphs that update regularly and show you the metrics you care about.

The final product isn’t always a static graph, though. More and more, data teams are working on ways to empower non-technical stakeholders to explore data themselves and answer their own questions. 

Tools for product analytics

As you can (hopefully) tell, there’s a lot that goes into that graph showing you how many users your product has, from raw events to transformation and visualization. Though most modern companies will do something like what’s outlined above, there are better and better tools on the market that automate pieces of this process and make them more accessible for non-technical teams. 

There is no substitute for a good data team, and there are always going to be questions that are too deep for SaaS tools to help you answer. That’s why even if you use some of the tools below, chances are you will still have a data team working on the steps outlined above. There are, of course, many many more tools I haven’t mentioned here – leave a comment with any of your favorites.

Segment – instrumentation and routing

Segment helps with multiple parts of the process, from a library for tracking events to routing those events to your warehouse (and other destinations). 

Segment was acquired by Twilio in 2020, so things are a bit up in the air there.

Amplitude – visualization and exploration

Amplitude is a data visualization platform made specifically for events. You can ingest your product events and build charts, dashboards, etc. It’s made mostly for non-technical teams, so you don’t need to write SQL to analyze your data; but as mentioned above, using SQL to analyze event streams isn’t exactly a walk in the park. Personally, I like using Amplitude for questions on the simpler side of things.

The nice thing about Amplitude is it lets you avoid the transformation step for certain questions. E.g. If you want to know what the most popular pages visited in your product are, you can just do that in the Amplitude UI, instead of needing your data team to build a table with that information. As mentioned though, for more complex explorations, this is probably not the tool.

FullStory / HotJar – session recording

FullStory and HotJar are session recording tools: they look at the actions your users take and replay them to you, like you’re watching a video of their screen. Logistically, they’re not actually recording a user’s screen – that would be creepy – they’re instead looking at how the user’s actions impact the HTML of the page, and then rendering what we imagine that probably looked like when the user did it. 

​​Since it’s mostly HTML tagging under the hood, you can filter for things like which page a user visited, what they clicked on, etc.

I have not heard of teams building this functionality on their own, but if you know of any, leave a comment on this post!

PostHog – full stack, everything

PostHog is probably the most unique tool here, because it obviates most of the need for the entire process of building a product analytics practice. Their product extends from event capture all the way to visualization and exploration; they even have functionality around heatmaps and session recording, something you’d normally need to use Fullstory/HotJar for. Their bet is that product analytics should all happen in one place, and that non-technical teams should be empowered to answer almost all of their own questions, with little or no help needed.

PostHog supports all kinds of visualizations – like funnels, user paths, etc. – specifically built for product analytics. They even have support for feature flags, where you release a new feature to a segment of your user base to test how they react to it. Perhaps the most interesting piece of the puzzle: PostHog is open source, which means it’s completely free to run on your own infrastructure (all other products on this list are paid and expensive).

A technical background is a superpower for product managers. You’ll make better decisions, understand trade-offs, make more accurate estimates, and communicate with engineers with more confidence, while also creating more career opportunities for yourself. And now that AI is increasingly infused into products, it’s even more important that product managers understand the basics.

Building a strong foundation of technical skills improves your ability to bridge technical and business domains, uncover implementation issues before they’re in production, and improve your working relationships with your technical counterparts. This quick guide will help get you up to speed and start building your skills with hands-on practice.

Application architecture and APIs

Almost every software product is built with the same three parts:

• a client,

• a server, and

• a database.

  Things can get a lot more complicated as you grow, but these fundamental ideas stay the same.
  The client is your website or app that’s used by the end user.
  The server processes requests from the client.
  The database is your permanent storage of data.

Your client communicates with your server through an application programming interface, or API.

What is an API?

For most software product managers, we’re talking about web APIs. This is the front door to your server that allows other applications to access data or trigger events.

APIs are everywhere—Meta, Etsy, OpenAI, your bank, your doctor, the weather. These kinds of public APIs are meant to be used by developers outside of the company that built them. Anyone who has permission can start making API calls to get data, submit new information, or even interact with AI. 

APIs can also be private. These are intended to be used only within that application or product (versus by third parties).

As a quick example, let’s look at how Airbnb might fetch listings to show you when you search for Toronto.

Breaking this down, we can see that:

1. The client sends a request to listings at the URL api.airbnb.com and specifies the location as Toronto

2. The server gets the listings in Toronto from the database

3. The database returns listings to the server

4. The server returns listings to the client

Wait, so what exactly is the API?

Is the API the server? Kind of. As we said above, the API is the server’s front door—it’s the set of actions that are available to clients. The API (hosted on a server) will either process the request itself or pass it downstream to be processed.

Going back to our Airbnb example, in addition to fetching Listings, you could imagine the API also allows a client to access Users, Bookings, Prices, and more. We commonly call these endpoints or resources.

Let’s take a look at another example: OpenAI. OpenAI supports a wide range of actions via their API. You can create images, transcribe audio, and get chat responses. Here’s what their API endpoints look like:

OpenAI has organized their API into a few groups—images, chat, and audio. Within each, there are a few endpoints that allow you to take different actions.

What are some common patterns with APIs?

Once you understand a few basic patterns with APIs, you’ll be able to breeze through technical documentation in no time.

First, there are three main types of APIs:

• SOAP: This is a legacy design, still used mostly in health care and financial services.

• REST: This is the most popular design pattern; we’ll get into more details below.

• GraphQL: This is a newer design created at Meta, most commonly used in private APIs within your own application. It uses the same request types but in different ways.

As mentioned, REST is by far the most popular design for APIs. It supports many different request types, but there are two we’ll focus on today—GET and POST.

GET requests

GET requests do what you think they would: get stuff. Every time you open a web page, your browser sends a GET request to the server at the URL you’ve entered and the server returns some content to be rendered.

In its simplest form, this content can be an HTML file that your browser displays, or it can be a more complex format that needs to be interpreted, like JSON.

Let’s update our Airbnb example above to specify the request type.

Now we can see that we’re sending a GET request to the /listings endpoint. We also specified that we only want listings in Toronto.

Let’s break the request down into its components:

1. Base URL: The “home” for all API endpoints

2. Endpoint: The specific resource we want to access

3. Query parameter: Additional data passed in with our requests
   
   

   POST requests

   POST requests are used to submit new data to the server. For example, we would use a POST request on Open AI’s /images/generations endpoint to create a new image. 

   We would also include a body with the request, which contains data to be processed by the server. Here’s what that might look like:

As long as we have access, OpenAI will respond to this request with the cutest baby sea otter you’ve ever seen.

Remember that text-to-speech example with an AI-generated voice that we got from OpenAI’s API? Well, that was a POST request.

To generate this, we sent a POST request to https://api.openai.com/v1/audio/speech with the following body:

				
					{
    "model": "tts-1",
    "input": "Did you know subscribers to Lenny's Newsletter get exclusive discounts with partners? Pretty cool! And there's a Slack channel!",
    "voice": "alloy"
  }
				
			

In response to this request, the API sends back an MP3 file with the text-to-speech phrase.

How can I practice?

Want to try a request for yourself? Follow along to generate your own images using DALL-E and OpenAI’s API.

1. Sign up for an API account at https://platform.openai.com/signup 

2. Navigate to API Keys. Verify your account and create your first API key. Make sure you copy it!

3. Using Postman or a web-based API request tool (here’s a free one), make a POST request to https://api.openai.com/v1/images/generations

4. Add the following Headers

   1. Authorization: Bearer [your-api-key]

   2. Content-Type: application/json

5. Add the following body in JSON

				
					{
    "model": "dall-e-3",
    "prompt": "A cute baby sea otter",
    "n": 1,
    "size": "1024x1024"
  }
				
			

6. Send your request!

This post will transform how you build products, come up with new ideas, and operate as a PM.

Whether you’re already deep into AI tools or just getting started, you’ll learn what tools you should be paying attention to, which tool to use when, and how to get unstuck when you run into an issue. You’ll find a collection of battle-tested prompts, real-world examples, and a step-by-step guide you can put into practice immediately. Imagine being able to turn Figma designs into a working app with a few clicks, or turn your PRD into a working prototype in minutes. This is all possible, and you’ll learn how.

If you haven’t been paying close attention over the past six months, you may have missed the rise of tools like Cursor, Replit Agent, v0, Bolt, and other new cutting-edge AI tools that allow you to build working apps in minutes. For example, it took me 10 minutes to build this 2-D tank game (with an AI opponent included), merely using this series of prompts:

• “Build a 2d tank game with an AI opponent.”

• “Add collision for the shot when it hits a tank.”

• “When health hits zero, play an animation and reset the game.”

• “Improve the acceleration for player movement.”

• “Make it so holding down the space bar has a timer to shoot a 2nd time.”

• “Add power ups to the map.”

Pretty cool. But what’s cooler is that you can use these tools to build functional prototypes from a Figma design, convert a rough hand-drawn sketch to a working app, translate a PRD document into an interactive prototype, or even build a usable internal tool for your team, with no coding ability. In this post, I’ll cover the basics of AI prototyping, show how to get good results out of the most popular tools, and walk through an end-to-end example of building a prototype in less than 10 minutes.

Choosing your tooling

Current AI development tools come in three types:

1. Chatbots (e.g. Claude, ChatGPT): The AI tools you probably know, which can also write and explain basic code

2. Cloud development environments (e.g. Replit, Bolt, v0, Lovable): Full-stack platforms that can build and run your apps in the cloud

3. Local developer assistants (e.g. GitHub Copliot, Cursor, Windsurf, Zed): Development environments (i.e. IDEs) that help you write code with the help of AI

Let’s review the most popular tools in each category to see what they can do and what we can build.

Chatbots (ChatGPT, Claude)

Best for: Prototypes that are just one page and don’t have complex design requirements, like calculators, flip cards, or data visualizations

Chatbots are capable of writing code in response to a question or prompt.

A prompt like “Build me a calculator with React” results in the following:

If you want to run this code, ChatGPT requires you to copy and paste the code into your IDE and run it on your own computer.

Claude goes one step beyond ChatGPT’s abilities with their Artifact system. Artifacts allow you to run the code within Claude’s interface and deploy to a shareable link. An unfortunate limitation is that you can’t make any direct edits to the code, so you’re entirely reliant on using prompts to make code changes.

(Perplexity is another tool you’ve probably heard of, which feels similar to chatbots but is more focused on search and not as useful for creating apps. It can write basic code because it’s built on top of other AI models like ChatGPT and Claude, but I wouldn’t recommend it for this use case.)

Remember that chatbots can write code for any part of our stack (client, server, database) but can’t host your code (deploy) for us. They also can’t create complex prototypes with multiple pages, and it’s difficult to change the code directly. As a result, these tools are best used for very simple one-time prototypes—which sometimes is enough to get the job done. Think of chatbots when you’re looking to create a very simple landing page, individual inputs like a date picker, or small apps like a to-do list.

Cloud development environments (Replit, Bolt, v0, Lovable)

Best for: Prototypes with more than one feature, specific design requirements, or many pages

Cloud development environments are one big step up from chatbots. These tools handle all the tasks required to turn your ideas into an actual working product. They can help you build end-to-end features, handle the backend infrastructure necessary to run your prototype, allow multi-file edits with agentic workflows, and take on more complex tasks across your codebase such as updating your database schema.

One of the key differentiators among the various cloud development environments is hosting. Every software product is built with three parts: a client, a server, and a database. The client is what the user interacts with (often written in JavaScript), the server processes requests from the client to retrieve data or integrate with other products (often written in Node.js, Python, or Java), and the database is your permanent storage of data. Making prototypes that have real features requires you to host both your client and server code, and may require a database to power the app.

One of the most popular tools today, v0, is capable of writing and hosting both client and server code. By default it uses specific frameworks called Next.js and Shadcn UI to do so (both were created by Vercel, the same company that owns v0). v0 can deploy your code and run backend servers—plus, one of its strongest features is that it has great styling as a default. Here’s a basic CRM I built in v0 with the prompt “Build me a basic CRM.”

Bolt is very similar to v0 in that it can also generate and deploy both client and server code. But a key difference is where the server runs. With v0, you deploy to real cloud hosting infrastructure, whereas Bolt runs the server code directly in the user’s browser. This means Bolt cannot natively support prototypes that need user identity like logins or accounts, multi-user interactions such as chat or collaborative workspaces, secure data operations like payment processing, or persistent data storage between sessions, because an isolated copy of the server is created on each user’s device. You can make up for this by integrating with external products like Supabase that offer servers and databases.

Here’s a basic CRM I built with Bolt, using the prompt “Build me a basic CRM.”

Another popular tool is Replit. Replit allows you to build full-stack applications, including a client, server, and database. It can build web apps using both JavaScript and Python frameworks and particularly excels at building internal admin tools (e.g. file conversion, job applicant tracking) and data-driven applications (e.g. image resizing, multi-page dashboards) with simple UIs.

I use Replit whenever I need a fully functional back end or I want to use Python code. I’ve used it to build an MP4-to-GIF converter and a Substack image resizer—both tools I use weekly.

Here’s a basic CRM I built with Replit, using the prompt “Build me a basic CRM.”

Finally, we have Lovable. It’s the newest of the bunch. Lovable is most similar to v0 and Bolt—it excels at generating websites, and uses JavaScript frameworks like React and Next.js. Its differentiation comes from its integrations with other popular tools. Lovable can connect to a GitHub repository, automatically add authentication and databases with Supabase, and help you connect to AI providers like Anthropic and OpenAI. All of these features make it one of the best AI coding tools for building products you actually want to use in production.

One major drawback of Lovable is the lack of a code editor. To edit code, you have to ask the agent with prompting. This can make it difficult to debug issues directly in Lovable. I often find myself starting a new feature here but moving over to Cursor to resolve problems.

Here’s a basic CRM I built with Lovable, using the prompt “Build me a basic CRM.”

To recap:

• Choose v0 for beautiful designs by default

• Choose Bolt for quick prototypes with flexible designs

• Choose Replit for internal tools or products that store or transform data

• Choose Lovable for building production apps that benefit from integrations with your current tools

Regardless of your choice, cloud development environments all support building more complex applications than chatbots, with the ability to deploy to the cloud and easily share updated iterations over time.

Local developer assistants (GitHub Copilot, Cursor, Windsurf, Zed)

Best for: People who know how to code and are working on serious applications they want to ship to production

The final type of AI development tool is local developer assistants. These products are targeted toward people who know how to write code. Tools like Cursor and GitHub Copilot can take prompts in a similar fashion to Claude but can then generate and apply changes within your own codebase and development environment (IDE). These tools do more than autocomplete—they can now write most of your code just using prompts.

For example, I built this presentation app (with live Q&A and polls!) in about 10 days using Lovable and Cursor. I started the app in Lovable to build basic features quickly, synced my code to GitHub to allow editing in other tools, and made final changes and fixed bugs with Cursor. This application uses authentication, databases, real-time updates, and more.

Ten days may sound like a lot, but most of that time was spent resolving bugs and troubleshooting issues—something Cursor excels at compared with other tools, and something I get into a couple of sections down.

GitHub Copilot is more popular in enterprise environments, as it comes from a trusted vendor, Microsoft. It supports multi-file changes from prompts, code explanations, and more. I’ve found it works best when given very specific direction and does not perform as well as Cursor at more general instructions. For example, when I asked for a new feature without providing context, Copilot re-created components of my app that already existed, whereas Cursor modified my existing files directly.

Two tools I haven’t spent as much time with but people are excited about are Windsurf and Zed. Windsurf is another IDE that can suggest multi-line changes to files and suggest commands such as moving files on your behalf. It excels at working on larger, more complex codebases. Zed is a highly performant editor built with a variety of productivity features such as prompt libraries, slash commands, and keyboard shortcuts for common actions like applying AI-generated code.

Building your prototype

Now that we’re familiar with the basic tools, let’s build some prototypes. The two most common prototyping use cases for product managers are:

1. Converting an existing design to a functional prototype

2. Building an idea into a prototype from scratch

Converting a design to a functional prototype

Let’s turn the following design for Airbnb’s home page into a working prototype. Say you want to use this prototype to explore a new feature, such as a price filter.

Here’s our initial design:

I chose Bolt for this task, as it’s better at building off a pre-existing design and we don’t need the backend database provided by Replit. Here’s a prompt I used (which you can copy and paste). Make sure to include a screenshot of the design!

“Build a prototype to match this design. Match it exactly.”

Here’s how this experience looks in Bolt:

Next, we’ll add our new price filter feature. Notice in the prompt below how I describe every feature in detail. One pro tip with prompting these tools is to be hyperspecific when describing changes for your subsequent prompts, as it helps the AI pinpoint what should change.

“Implement an inline price filter as a component of the search bar. It should appear next to ‘Add guests’ in its own section.

Selecting the input should pop up a price filter with minimum and maximum values. The background of the pop-up should be white and should cover elements beneath it.”

This is a great starting point. Let’s extend this feature with a slider for the minimum price.

“Can you add a price slider? It should have a blue line and a black node. Sliding the node should modify the minimum price.”

We now have a functional initial prototype of your product idea within 10 minutes! Without any coding skill. Incredible. We could even continue to improve this prototype filter (e.g. showing listings updated in real time as you adjust the price). Check out the prototype here.

Building a prototype from scratch

If you’re like me, your design skills are probably not good enough to create the initial design we used in the prior example. Luckily, you can build a prototype using existing patterns and components from free and publicly available design systems like Tailwind or Shadcn UI.

Let’s build a quick CRM with Bolt, then add a new feature to it. Let’s say we’re considering adding a feature that automates email outreach directly from our CRM and want to gather customer feedback about this feature before building it, by showing potential users a prototype of what it might look like.

We can quickly put together a v1 with the following prompts:

“Create a comprehensive customer relationship management (CRM) system.”

Again, let’s take a moment to pause here. We just created a working prototype of a CRM in less than five minutes—something that would have previously taken weeks of an engineer’s time. Unbelievable.

Let’s keep going and see how we add a new feature.

“Please implement a mock AI email writer. This should be accessible from the left nav.”

Again, less than five minutes to be able to play with a new feature idea. Just think about how many ideas you can explore and how quickly you can bring them to market. AI prototyping tools have been available for less than six months and have already absolutely changed the speed at which teams can ship.

Now we can take our prototype and get customers’ direct feedback without wasting time developing an initial version. This approach can speed up your discovery process by getting interactive examples in your customers’ hands as early as possible.

Each AI tool will have very different outcomes based on their default settings and how specific you are. Here’s the exact same example with the same prompts using v0:

Common use cases and prompt templates

Here are a few good templates you can use to get started with each of the tools I’ve highlighted:

Task #1: Build a prototype from an existing Figma design

Prompt:

Build a prototype to match this design. Match it exactly. Use Tailwindcss.

Match styles, fonts, spacing, and colors.

[Include a single screenshot from Figma]

Tool: Bolt

Example: Deployment manager

In today’s fast-evolving tech landscape, the API-first approach is making waves across industries. This isn’t just a trend—it’s a transformative strategy that’s reshaping how companies design, develop, and deliver software solutions. The data speaks for itself: more than 60 API-first companies in 2022 raised $50 million or more in funding, while API-first businesses collectively raised $14 billion. By 2030, experts predict that we’ll see around 1.7 billion active APIs, underlining their central role in the digital economy.

But what does “API-first” actually mean? Why is it so important for businesses today? And how can adopting this approach give organizations a competitive edge? Let’s dive into the technical and business dimensions of API-first development and uncover why it’s becoming a cornerstone of modern software architecture.

What Does API-First Mean?

To understand the API-first approach, it’s essential to grasp the concept of an API itself. An API, or Application Programming Interface, is a set of rules and protocols that allows different software applications to communicate with each other. APIs are like the bridges connecting apps, enabling them to exchange data and work together.

One of the most familiar examples is social media logins. Ever used a website that allows you to log in using Google, Facebook, or Apple? That’s an API at work, seamlessly connecting your social media account to the website you’re accessing.

In the API-first development model, APIs aren’t treated as afterthoughts. Instead, they are prioritized from the very beginning of the development process. This approach is rooted in the idea that APIs should be at the core of how software is designed and built, ensuring that applications are not just functional but also scalable, interoperable, and ready for integration.

The API-First Approach: Technical and Business Perspectives

The API-first methodology benefits both developers and businesses. Let’s explore these advantages from two key viewpoints.

1. Technical Perspective: A Developer’s Dream

From a technical standpoint, the API-first approach is about creating software that’s modular, interconnected, and future-proof. Here’s why it matters:

• Seamless Integration and Connectivity
  APIs act as the glue between different software components. Whether you’re building microservices or integrating third-party platforms, APIs ensure smooth communication and data sharing. This modularity makes it easier to assemble complex systems from smaller, reusable components.

• Cross-Platform Compatibility
  APIs are designed to work seamlessly across devices and platforms, ensuring that software can cater to diverse user bases. Whether it’s a mobile app, a web application, or even IoT devices, APIs enable consistent experiences.

• Scalability and Flexibility
  As businesses grow, so do their software requirements. APIs enable developers to scale systems by adding or updating components without disrupting the entire ecosystem. This adaptability is crucial in accommodating larger user bases and evolving business needs.

• Enhanced User Experiences Through Data Integration
  APIs can pull data and features from various sources, creating applications that deliver richer, more dynamic user experiences. For example, a travel app can combine flight data, hotel availability, and weather forecasts—all through APIs.

2. Business Perspective: Unlocking New Opportunities

The API-first approach isn’t just a technical strategy—it’s a powerful business enabler. Companies like Stripe, Twilio, and AssemblyAI have demonstrated how API-first can drive growth and innovation. Here’s how it works:

• Monetization Opportunities
  APIs can be a revenue stream in themselves. Companies can charge for API usage, offer premium features, or establish partnerships with other businesses that rely on their APIs. For instance, Stripe’s APIs power payment systems globally, generating significant revenue.

• Expanding Market Reach
  APIs enable third-party developers to build applications and services on top of your platform. This creates a developer ecosystem and helps your business expand its market presence. Think about how platforms like Twilio have become indispensable to developers building communication tools.

• Faster Time-to-Market
  With APIs, companies can reuse existing software components, reducing development time. This allows startups and businesses to bring products to market faster, staying ahead of competitors.

• Competitive Edge Through Integration
  In today’s software landscape, customers expect tools that integrate seamlessly with others. API-first companies are better positioned to meet this demand, providing superior integration options that attract and retain clients.

• Agility and Innovation
  By prioritizing APIs, companies can quickly adapt to changing market conditions. APIs enable businesses to iterate and innovate faster, ensuring they stay relevant in dynamic industries.

Key industries for API-first development

While the API-first approach can bring value across various domains, here are sectors where this approach is especially beneficial. 

Healthcare

In healthcare, APIs are crucial for bringing together multiple players and ensuring the data is shared between them in an accessible and secure manner.

• Healthcare organizations must securely transmit electronic health records (EHRs) and patient information among various providers, insurers, and systems. API-first integration ensures the seamless and secure exchange of medical data, improving patient care coordination. 

• More than that, APIs enable the integrating of disparate healthcare systems, such as electronic medical record (EMR) systems, telehealth platforms, and diagnostic equipment. APIs also empower developers to create innovative healthcare applications, including mobile health (mHealth) apps, remote patient monitoring solutions, and telemedicine platforms. These applications can transform the way patients access care and manage their health. 

For example, SmartHMS, an EHR provider, offers APIs that allow healthcare organizations to integrate patient data seamlessly. This enables healthcare providers to access patient records, exchange information securely, and improve care coordination.

Banking and finance

The financial industry relies heavily on data and transactions, making the API-first approach beneficial for improving services and operations.

• API-first strategies are at the heart of open banking initiatives, enabling financial institutions to expose their services and data to third-party developers securely. This encourages the creation of new financial products, and enhances customer experiences.

• APIs facilitate real-time access to financial data, enabling customers to view account balances, conduct transactions, and receive alerts instantly. This real-time capability is essential for online banking and mobile banking applications.

• APIs can be used for fraud detection and prevention services, enhancing security in financial transactions, and enabling rapid detection of suspicious activities and help mitigate fraud risks.

The most notable example here is, of course, Razorpay. Razorpay's API-first approach revolutionized online payments. Its APIs enable businesses to integrate payment processing into their websites and apps seamlessly, simplifying the checkout process and enhancing the user experience.

Travel and hospitality

This industry relies on seamless customer experiences, making the API-first approach invaluable for improving services and expanding offerings.

• APIs allow travel companies to easily connect users with airlines, hotels, car rental services, and other providers, streamlining the booking and reservation process, and offering customers a one-stop-shop for travel arrangements.

• By leveraging APIs, travel companies can access customer data and preferences, enabling them to offer personalized recommendations, travel itineraries, and loyalty rewards. This personalization enhances the overall travel experience.

• APIs enable real-time access to pricing and availability information. Travel companies can adjust pricing based on demand and market conditions, maximizing their revenue and competitiveness.

Amadeus provides APIs for travel booking and reservation systems. Airlines, hotels, and travel agencies can integrate their APIs to offer comprehensive travel booking and management services, improving the traveler's experience.

API-first approach and UX design

When you think about launching a SaaS business, you should always keep user experience (UX) at the forefront. And going for an API-first design is no exception. 

The API-first design and development allows for a more personalized and seamless user experience. For example, by combining API functionality with thoughtful UX design you can better address users’ needs and preferences. APIs provide the data and functionalities necessary for a feature-rich application, while UX design ensures that these features are intuitive and user-friendly. 

UX designers can also leverage APIs to streamline user journeys. For example, integrating geolocation APIs can enhance the user experience by providing location-based services, such as finding nearby stores or restaurants, letting users achieve their goals much easier.

Finally, the collaboration between API development teams and UX designers promotes an iterative approach to product development. Developers and designers rely on user feedback, as well as on each other's insights to continuously improve the product.