One of the most common insecurities for product managers (PMs) is their technical skills. While you don’t need to be an engineer, having a solid understanding of how software works can help you communicate effectively with your team, manage risks, and make better product decisions. Let’s break down the technical jargon and concepts you need to know to feel more confident in 2025 and beyond.

How Internet Software Works: The Core Components

All internet-based software products follow the same foundational architecture. They consist of three main components:

1. Client: This is what your users interact with directly. Examples include websites, mobile apps, smart devices like Tesla cars, or even smart fridges.

2. Server: This is the brain of the operation. It processes requests from the client, fetches or stores data, and performs actions like syncing your Google Calendar.

3. Database: This is where all the data is permanently stored. Think of it as the filing cabinet for your application.

Each of these components can be broken down into smaller, more specialized parts. Building scalable, well-organized software requires careful planning and execution. Let’s dive deeper into each of these components.

Frontend: What Users See and Interact With

The frontend is the part of the application that users interact with directly. It’s the code that runs on the user’s device, whether it’s a browser, a mobile app, or a smart device. Frontend development involves creating intuitive, responsive, and visually appealing interfaces.

Key Technologies in Frontend Development

• HTML (HyperText Markup Language): This defines the structure of a webpage. It outlines where elements like headings, paragraphs, and buttons appear.

• CSS (Cascading Style Sheets): This controls the styling of the page, including fonts, colors, spacing, and layouts.

• JavaScript (JS): This adds interactivity to the page. For example, it allows animations to play, forms to be submitted, or new data to be loaded without refreshing the page.

Frontend Frameworks

As web applications have grown more complex, developers have turned to frameworks to streamline their work. These frameworks provide pre-built tools and components, making it easier to organize code and build scalable applications. Popular frontend frameworks include:

• React: Developed by Facebook, React is widely used for building dynamic user interfaces.

• Vue: Known for its simplicity and flexibility, Vue is a great choice for smaller projects or teams.

• Angular: A robust framework maintained by Google, Angular is ideal for large-scale enterprise applications.

How Frontend and Backend Communicate

The frontend communicates with the backend using HTTP requests. These requests are sent to specific endpoints (like a mailbox) on the server. Common types of HTTP requests include:

• GET: Retrieve information (e.g., loading a user’s profile).

• POST: Submit information (e.g., creating a new account).

• PATCH/PUT: Update information (e.g., changing a user’s password).

• DELETE: Remove information (e.g., deleting a post).

Backend: The Engine Behind the Scenes

The backend is where the magic happens. It’s the part of the application that users don’t see but is critical for processing data, handling business logic, and ensuring everything runs smoothly. Backend code is not distributed to users—it runs on servers hosted in the cloud.

Cloud Hosting Providers

Most modern backend systems are hosted on cloud platforms like:

• Amazon Web Services (AWS): The most widely used cloud provider, offering services like EC2 (servers), S3 (file storage), and Lambda (serverless functions).

• Google Cloud Platform (GCP): Known for its data analytics and machine learning capabilities.

• Microsoft Azure: A popular choice for enterprises, especially those already using Microsoft products.

These platforms allow you to rent computing resources like servers, databases, and storage, which can be scaled up or down based on demand.

Key Backend Components

• API Gateways: Act as the front door to your backend, routing incoming requests to the appropriate services.

• Caches: Temporary storage for frequently accessed data, speeding up response times.

• Queues: Help manage high volumes of requests by storing them until they can be processed.

• Content Delivery Networks (CDNs): Store static files like images and videos in multiple locations worldwide to reduce load times for users.

Scaling: Handling Growth Without Breaking

As your product grows, so does the demand on your backend. Scaling is the process of adding resources to handle increased traffic and ensure your application remains fast and reliable.

Types of Scaling

1. Horizontal Scaling: Adding more machines to distribute the load. For example, going from 1 server to 10 servers.

2. Vertical Scaling: Upgrading existing machines with better hardware, such as more memory or faster processors.

Challenges of Scaling

Scaling isn’t just about adding more resources—it’s about ensuring the entire system can handle the increased load. Common challenges include:

• Bottlenecks: One component (e.g., the database) may become a bottleneck if it can’t handle the increased traffic.

• Cost: Scaling can become expensive, so it’s important to optimize resources.

• Complexity: As you add more components, the system becomes harder to manage.

Tools for Scaling

• Pub/Sub: A messaging system where one server publishes a message, and others subscribe to it.

• Kubernetes: A tool for automating the deployment and management of containers (packaged applications).

• Batching: Grouping multiple requests together to improve efficiency.

• Rate Limiting: Controlling the number of requests a user or service can make in a given time period.

Common Backend Terms Every PM Should Know

• Pubsub: A queue. A message is sent in to the queue (published) and one or more servers can receive it (subscribe)

• EC2: A product offered by AWS to run a server (Elastic Compute Capacity)

• S3: A product offered by AWS to store files (Simple Storage Service)

• SQS: A product offered by AWS to run a queue (Simple Queue Service)

• SES: A product offered by AWS to send emails (Simple Email Service)

• Lambda: A product offered by AWS to rent a server for a very short period of time

• Containers: Document everything you need to run your application in one file

• Kubernetes: Deploy a container to a server and manage all your servers automatically

• Batching: Grouping requests together instead of sending them one at a time

• Rate limit: The maximum number of requests that can be processed in some duration

• Microservice: A set of resources (server, database, queue) that are built to do a specific task, like handling billing, notifications, recommendations, etc.

Why This Matters for Product Managers

Understanding these technical concepts isn’t about replacing your engineering team—it’s about empowering yourself to:

• Communicate Effectively: Speak the same language as your engineers and align on goals.

• Manage Risk: Identify potential technical challenges early in the product development process.

• Make Informed Decisions: Evaluate trade-offs between different technical approaches.

The TL;DR

A relational database is sort of like Excel, but for developers: it’s how applications store and analyze data.  

• Relational databases like MySQL and PostgreSQL are the default software for storing data

• Relational DBs are all about schemas: rules for how data is stored and organized

• Inserting data into relational DBs follows the ACID specification

• SQL is a type of programming language that lets you pull and analyze the data in your relational DB

Relational DBs power at least part of almost every single application that you use in your day to day. So read this.

What’s a relational database?

→ First: what’s a database in general

A database is just a place to store data; even Excel is technically a database. But when developers talk about databases, they’re usually referring to specialized software that’s used to store data from applications. When you sign up for Facebook, a new entry gets added to Facebook’s users database with a bunch of information about you; every time you sign in after that, that data gets pulled and used (this is part of what a backend is).

There are thousands of different databases out there, all optimized for different types of applications, some free and some paid. Some popular ones you might have heard of: MySQL, PostgreSQL, and MongoDB. 

🚨 Confusion Alert 🚨

People use “database” to refer to a lot of different things: most of the time it’s our topic (programmatically accessed data stores), but you might also hear about “my database of top CEOs” or something like that, which is probably a spreadsheet. Just listen for the context and you should be fine. 

🚨 Confusion Alert 🚨

→ Second: what’s a relational database

Relational databases are the most popular kind of DBs, and they’re pretty simple: they add rigidity and rules around how data is stored. To understand what that means, let’s imagine we’re moving across the country (yes I’m still crying, no I don’t want to talk about it). You’ve got a ton of stuff in your house: clothes, furniture, random keychains that you can’t let go of, and kitchenware, among other things. How do you store / pack all of that?

• You could throw everything in random boxes, like back in college 

or

• You could organize things by size or category, and pack and label specific boxes

There’s a clear tradeoff here between speed and efficiency. Packing properly takes a lot more time, but allows you to make better use of space and more easily find your stuff later. It turns out that databases have the exactly same tradeoff, and relational DBs are like packing properly instead of throwing stuff in. 

In relational databases, you specify everything about your data up front: 

• What type it is (text, number, yes/no, etc.)

• How it’s organized (different tables, how those tables are related to each other, etc.)

• Names for everything (columns)

All of this together is called a schema, and it’s the basis of relational DBs. But why are they called relational?

Relations, take 2

A relational database is called what it is because of the relationships it models between your data.

In a relational DB, data is organized into tables with rows and columns: you might have a table for data about your app’s users, as well as a table for your app’s orders. Those tables are related, because each order has a user that made that order. Big applications can have hundreds or thousands of those tables, and you need to define the relations between them for it to work well. 

In this diagram (and, I’m guessing, in Amazon’s internal systems) there’s a users table and an orders table: the relation between those tables is that each order has one user who created it. 

There’s a good reason why things are designed this way instead of throwing all of your data into one giant table: it makes inserting and pulling data way easier and more efficient. In the early days of the internet, when this technology was just starting out, it was incredibly expensive to store and retrieve data. So people spent a ton of time on optimizing it: making sure the same data is never stored twice (this is called normalization), finding the most efficient way to store different data types, and all of these low level things that are beyond my pay grade. 

Transactions and SQL

Just in case there weren’t enough acronyms yet, I gotchu.

→ ACID transactions

Modern software applications need to be fast and make few or no mistakes. Facebook can’t mess up your login, or mistake you for another user: that’s why relational DBs have special features around transactions. A transaction is just an interaction with the database, either inserting data into it or querying and pulling data from it. Transactions can go wrong all the time for a ton of different reasons, but there’s a body of standards called ACID (broooo) that helps avoid them. 

The specifics of ACID aren’t too important. Just keep in mind that if you see a phrase like “ACID compliant transactions” it’s referring to these standards.

→ SQL

Pulling data from a database is called querying, and there’s a series of special programming languages that let you do that called Structured Query Language (SQL). There are all different types and flavors of SQL (depending on which relational DB you’re using), but they’re all pretty similar. A typical SQL statement might look like this:

				
					SELECT 
    first_name,
    last_name,
    address,
    last_order_date
FROM users
WHERE signup_date > '2019-01-01'
				
			

SQL is pretty human readable – even if you don’t know it, you can guess what we’re doing here – and it’s a really useful thing for everyone to know, even if you’re not en engineer. 

☕️ Dev Culture ☕️

Software developers need to know basic SQL for building their apps, but the biggest SQL experts are usually Data Scientists and Analysts: their entire days (I was one!) are spent querying and manipulating data from databases. For that and a few other reasons, some developers treat SQL as a second-tier, not “real” programming language.

☕️ Dev Culture ☕️

Databases are the unsung heroes of modern technology. They power everything from your favorite social media apps to the analytics dashboards that businesses rely on for decision-making. But with over 300+ databases available, each designed for specific use cases, choosing the right one can feel overwhelming. Whether you're a  seasoned developer or an aspiring PM, understanding databases is crucial for building efficient, scalable, and reliable systems.

In this guide, we’ll explore databases in depth, using relatable metaphors, real-world examples, and clear explanations to help you understand how they work, why they matter, and how to choose the right one for your needs.

The Flour Metaphor: Databases as Ingredients

Let’s start with a metaphor: baking bread.

When baking, the type of flour you use determines the texture, flavor, and structure of your bread. High-protein flour (like bread flour) creates a chewy, dense loaf, perfect for pizza or bagels. Low-protein flour (like cake flour) makes light, flaky pastries. All-purpose flour works for everything, but it’s not ideal for specialized recipes.

Databases are like flour. Each one is designed for specific use cases. You can use any database for anything, but choosing the right one ensures the best possible bread outcome. To understand which database to use, ask two key questions:

1. What does the data look like?

2. What do you need to do with it?

Let’s break these down.

1. What Does the Data Look Like?

Databases are optimized for specific types of data. While you can store any piece of data in any database, the way data is inserted and retrieved matters most. Postgres is relational, but supports big blobs of text. Nothing is stopping you from storing your user data in Elastic. Nevertheless, many databases are designed with specific types or shapes of data in mind.

Here’s a closer look at how data shape influences database choice:

Structured Data

• What it is: Data with a clear, predefined schema (e.g., tables with rows and columns).

• Example: User profiles, orders, or inventory.

• Best databases: Relational databases like Postgres, MySQL, or MSSQL.

Unstructured or Semi-Structured Data

• What it is: Data without a fixed schema (e.g., JSON documents, logs, or social media posts).

• Example: Tweets, sensor data, or product reviews.

• Best databases: NoSQL databases like MongoDB, Cassandra, or Elasticsearch.

Interconnected Data

• What it is: Data with complex relationships (e.g., social networks, recommendation engines).

• Example: Friend connections on Facebook or product recommendations on Amazon.

• Best databases: Graph databases like Neo4J.

Time-Series Data

• What it is: Data indexed by time (e.g., stock prices, sensor readings).

• Example: Hourly temperature data or daily financial transactions.

• Best databases: Time-series databases like InfluxDB or TimescaleDB.

2. What Do You Need to Do with the Data?

Different types of data – and even similar types of data for different use cases – get used in different ways. How you use the data determines the database you need. Here are some common use cases and the databases that excel in them:

OLTP (Online Transactional Processing)

These are the databases that you use for the core data in your application, think your user data. Data is being read and inserted constantly. Queries are usually small and quick. Data integrity is key.

• What it is: Real-time, transactional data processing (e.g., user actions, payments).

• Example: Adding items to a shopping cart or updating a user profile.

• Best databases: Relational databases like Postgres or MySQL.

OLAP (Online Analytical Processing)

These are the databases that you use for analytics and data science, think Snowflake etc. Data is being read and inserted sporadically. Queries are usually large and complex. 

• What it is: Complex queries on large datasets for analytics or business intelligence.

• Example: Calculating monthly revenue or analyzing user behavior.

• Best databases: Data warehouses like Snowflake, BigQuery, or Redshift.

Operational Use Cases

• What it is: Logs, monitoring, caching, or internal processes.

• Example: Storing server logs or managing user sessions.

• Best databases: Elasticsearch for logs, Redis for caching, or Prometheus for monitoring.

The fact that users of the database need to do different things with the data inside of it necessitates storing said data differently. 

OLTP vs. OLAP is one example, but there are many different ways to break down those use cases. Some databases have special UIs on top for analysis (like Kibana and Elastic). Others are built for massive scale from the start. Some are made to handle many different situations decently well. No matter what database you’re dealing with, thinking about what kind of data it stores and how you need to use that data is the key to understanding each.

3 major categories of databases and what they do

Crude as it may be, I find it useful to think of 3 major categories of databases. Some databases can overlap and cross categories. For example, there are NoSQL databases built for powering your app, but there are also NoSQL databases meant for in-memory stuff (e.g. Redis is technically NoSQL). Here are the major ones:

1. Databases that power a user-facing app – production databases that store the data you need for your app to run.

2. Databases that power analytics – databases for analysis, machine learning, and anything a data team does.

3. Databases that power operations – databases for monitoring, logs, security, and any internal processes that enable the above.

Let’s run through each category and cover a few examples for each.

1. Databases That Power User-Facing Apps

Databases that power a user-facing app – often referred to as production databases or production data stores – are where developers store and query whatever data their app needs to run. Any “data” you’re seeing on your screen sits here.

• If you’re Twitter: tweets, user profiles, trending topics, DMs

• If you’re Gmail: emails, settings, spam filters

• If you’re Amazon: orders, users, credit cards

Production databases are built to support small and quick queries. The most important thing is data integrity and reliability – you don’t want to lose an order or a credit card.

A. User-facing DBs / Relational Databases

1. Relational databases: are the database OGs. They’ve been around for pretty much as long as databases have been around, and are the default choice today for storing the basic data your application needs to run. Data is stored in a very structured format with rigid definitions of tables, columns, and how tables relate to each other; that way, queries can run quickly and reliably.

2. Postgres is an open source relational database initially released back in 1997. It’s one of the standard “boilerplate” choices for teams getting started on a new app. You can run Postgres yourself on a server, or pay someone like Amazon to run it for you.

3. MySQL is also an open source relational database, initially released in 1995. It’s the other database most teams reach for, although since Oracle bought Sun (and ergo MySQL), there are legal ramifications to modifying the MySQL code for commercial use. In terms of MySQL vs. Postgres, the short story is that they’re very, very similar but have a few important differences with how they handle data types, storage engines, and a few other things.

4. MSSQL is Microsoft’s proprietary SQL database. It’s popular among very large companies but not for smaller startups.

5. Oracle was the first commercial relational database ever. Today it’s got the reputation of being the slowest moving company in California, but nevertheless powers the apps of tons and tons of Fortune 50 companies. Like MSSQL, this is a database for massive organizations and not startups.

6. MariaDB is another open source relational database, from the original creators of MySQL. They (in a somewhat cheeky fashion) seem to be pretty pissed about Oracle limiting contributions to MySQL, so they focus on how open the community is and how the software will never require a license.

7. SQLite is a software library that lets you run a small database inside of your application. It’s used in one way or another in tons and tons of apps, usually as an intermediate database as opposed to a core production store. The website says it’s on every single Android, iPhone, etc.

B. User-facing DBs / NoSQL Databases

1. NoSQL: databases remove the structure and rigidity from relational databases, and let you just dump data in there and worry about how it looks later. There are NoSQL databases for all different categories of database (analytics, operational) but these here are for production, user-facing use.

2. MongoDB popularized the NoSQL moniker and is the default choice if you’re looking for a NoSQL database to power your app. Their cloud product, Atlas, is the AWS RDS equivalent for basic NoSQL. 

3. Cassandra is a NoSQL database built for really big companies who need to store lots of data and retrieve it fast. Unlike MongoDB, which is built as a document database, Cassandra is oriented more around a columnar setup, which means data is stored in entire columns (like Snowflake, actually). Using Cassandra feels a lot more like using a relational database. Thank you to Neo Kim from the System Design Newsletter for pointing out that Cassandra is actually technically not a columnar data store, but instead implements the concept of a wide column family, which is slightly different.

4. DynamoDB is AWS’s proprietary NoSQL database. 

5. Firebase is a series of tools (acquired by Google back in the day) for building apps, targeted at smaller teams and focusing on simplicity. Their database is called Firestore and it’s a really popular option for a quick, just-get-me-started kind of DB. 

C. User-facing DBs / Graph Databases

Every company’s data is different, and for some it makes the most sense to model things as a sort of graph of interconnected nodes. Facebook famously runs their social graph on an in house graph database called Tao. The use cases for graph DBs are mostly user-facing, but they’re sometimes useful for analytical purposes too.

1. Neo4J is the most popular graph database. It’s open source but if you’re going to scale it up, you’ll need to talk to them about their enterprise license. Data in Neo4J is represented as nodes, and nodes can be connected to each other in different ways. Cyper is the name of their SQL-esque language for querying the graph.

2. Databases that power analytics

What a data team needs from their database is very different from developers working on an app. For data science and analytics work, the data you store is often redundant, there’s tons of it, and your queries join data from multiple tables at once. Data usually gets queried by someone sitting at a computer doing research, or a system building a pipeline, and gets inserted at regular intervals (twice a day, something like that). 

A. Analytical DBs / Data Warehouses

Data warehouses are (usually) relational databases for storing analytical data, like what your users have been doing, revenue by month, things like that. They’re optimized for big, long, multi-table queries. They’re usually relational databases in nature, although the implementation details can vary.

1. Snowflake is a cloud data warehouse for analytics. It’s columnar, which means that data is stored (under the hood) in entire columns instead of rows; this makes large analytical queries faster, so it’s a common choice for how to build analytical DBs.

2. BigQuery is also a cloud data warehouse for analytics. It’s very similar to Snowflake.

3. Redshift is also a cloud data warehouse for analytics. It’s very similar to Snowflake and BigQuery.

4. Clickhouse is an open source (!) data warehouse for analytics. It was originally developed at Yandex, and is getting more popular.

🤔 Why can’t I use Postgres as a data warehouse?

You can, people did for a long time, and some still do. But modern cloud data warehouses are just waaaaay faster for large queries on meaningful quantities of data. This is the thing about databases: you can use anything for anything, but there comes a point where you need a specialized tool for the job.

B. Analytical DBs / Data Lakes

1. Data Lakes are basically giant data safes. You throw whatever you want in there – structured, unstructured, big, small, organized, disorganized, whatever – and then worry about structure when it’s time to get the data out. Contrast that with a data warehouse, which like a relational database, has rigid structure around tables, columns, and data types.

2. S3 is AWS’s object storage solution – usually used for storing things like images and videos for applications – but is also commonly used as a data lake. You can’t “query” S3 in the way that you can a database, so you need to use a layer on top (like AWS’s Lake Formation) to intermediate.

3. Databricks, a company that I wrote about back in the day, sells a product they call “the lakehouse platform.” Under several layers of caked marketing makeup, it’s an open source data lake and storage layer built to resemble a sort of data warehouse; they’re trying to blur the distinction between the two.

Though it’s unlikely you’ll see it in production today, HDFS was a highly popular way to build a data lake before the cloud data warehouse era. It’s powered by Hadoop, one of the original frameworks for performing distributed computations on large groups of data. It was/is notorious for being very, very hard to set up and run.

C. Analytical DBs / GIS Databases

This is a bit of a niche one, but worth mentioning: there’s a class of databases (or in some cases, database extensions) that are purpose built for working with geographical data. GIS stands for Geographic Information System. In terms of the actual data being stored here, it can be anything from points and lines to complex 3D data; structures that don’t map (no pun intended) well to traditional database schemas.

1. PostGIS is a PostgreSQL extension for storing and working with geographical data. It’s a combination of features for storage, analysis (special geographical functions like intersections, measuring distances, etc.), and other miscellaneous stuff like geocoding.

2. Kinetica is an enterprise-focused database for storing geospatial and time series data.

3. Oracle Spatial Database is also an enterprise-focused database for storing geospatial data, from our friends at big red.

Databases that power operations

This is the category you’ll probably encounter least if you’re not an engineer. The 3rd category of databases covers data stores that developers use to power internal operations: monitoring application performance, storing logs and security information, improving application speed, or even intermediate layers between other databases.

Operational DBs / Key Value Stores

While pretty much all of the databases we’ve covered so far store permanent data – saved on a harddrive – there’s a class of databases that only keeps data in memory. They’re meant for ephemeral data that you need to store and use quickly, but you don’t mind if it disappears down the road. These databases are called key value stores (or KV stores) because the way they store data is like a dictionary, where each entry has a key.

1. Redis is the OG key value store. It’s an in memory database that teams use for a bunch of different stuff: building caches, managing authentication sessions, chat and messaging, and any other use cases that prioritize real time, very quick data retrieval. Redis is open source but you can also pay them to host it for you.

2. SingleStore (FKA MemSQL) is an in memory database like Redis, more focused on general purpose kinds of workloads than typical in memory database use cases.

Operational DBs / Time Series Databases

Time series databases are DBs built specifically for storing data on some sort of time frame: daily financial data, second-by-second sensor readings, hour-by-hour health checks on your servers, anything like that. 

1. Timescale is an extension to Postgres, sold as an independent cloud hosted database. It takes everyone’s favorite relational database and adds special time-series specific functionality like automatic partitioning and query optimizations.

2. Prometheus is an open source time series database with built in alerting and visualization.

3. InfluxDB is also an open source time series database.

Operational DBs / Logs and Search Databases

1. Elastic is a NoSQL database built for storing and searching through logs, or very granular records of server performance, API requests, and internal stuff like that. Elasticsearch is commonly used with Kibana, its sister data visualization tool.

2. Solr is also a NoSQL database built for storing and searching through logs.

Product analytics – what the hell is it? Why is Amplitude a public company? What do product managers even do?

This post will dive into how companies analyze what their users are doing in their product. We’ll cover what kinds of questions teams want answers to, how the data gets generated and moved, and what tools are out there to simplify things.

It all starts with decision making

The wonderful world of product analytics starts with knowledge, or at least the eternal yet fleeting desire for it. Startups and large companies alike want visibility into how their users use their digital products. They’re looking to understand things like:

• What are our most popular features?

• Which pages on our website convert the best?

• What areas of our product experience are confusing?

• What are common paths that users take to accomplish something?

Armed with this information, product teams can make educated decisions on what they should be improving and building. These kinds of analytics can also help catch bugs: if the number of users visiting a page drops precipitously one day, chances are something is wrong with that page, and needs fixing (or your analytics are broken 😏). 

Product funnels

Product and growth teams like to use the concept of a funnel to visualize the paths that a user takes through the product. The funnel is wider on top – representing actions that a lot of people take, like visiting a website, and narrower on the bottom, representing actions that fewer people take, like signing up for the product and using it. 

Each stage of the funnel (except the top one) has a conversion rate representing the percentage of people in the previous bucket that made it to the current step. Teams try to benchmark their conversion rates against other products to get a sense of whether they are too low (or hopefully, too high).

🚨 Confusion Alert 🚨

Even though it’s referred to as product analytics, this area has blurry overlap with marketing data. Growth funnels can include data on website visits, and which sources (referrals, organic search, etc.) are driving quality leads. Who is responsible for using this data varies; just don’t be surprised if you see traffic-related information – what you’d normally consider marketing – in this post.

🚨 Confusion Alert 🚨

User research and session recording

Numbers can only go so far in giving product teams a good sense of what their users are actually doing in the product. Sometimes, you just need to see how they navigate around and do their tasks. Traditionally, user research teams would find prospective or existing users, and actually observe them accomplishing common tasks in your app. For example, the user research team at Microsoft might pay users to join a study and ask them to build a basic financial model in Excel; they’ll pay close attention to exactly what the users do and what problems they run into.

More recently, tools like FullStory have popped up, which allow teams to literally record what users are doing in the product during their day to day, making all of this significantly easier. More on these tools later.

How it all works under the hood?

As with all things data, it’s a pain in the ass to arm your company with the data to answer all of the questions product teams have. You need to instrument your product to “record” what users are doing, get that data into your warehouse, transform it into a format that’s useful for analysis, and then build visualizations that give you the information you want. 

This section will walk through how that’s done, and the next will cover newer tools that are making things easier than they were.

Note: most tech companies today are using event-driven architectures like the one we’re about to describe. But there are other, worse ways of doing this.

Instrumenting the product

Step one is to actually create the data. And doing this is fairly simple, albeit custom: you just send what’s called an event any time a user does something in the product. Suppose we’d like to record every time a user clicks on the “save changes” button when editing their profile. We’d fire an event like this:

				
					analytics.track(
  {
    event_name: “save_changes_button_clicked”,
    timestamp: “2022-03-09 23:45:33”,
    context_page: “/profile/edit”,
    user_id: “239hg72eyrhdaadgdg99277hd7ve88d”
  }
)
				
			

The event gets a name, a timestamp, and some metadata: in this case, the context in which it took place (on the edit profile page), and the ID of the user who took the action.

Fun fact, you don’t need to take my word for this. If you’re logged into a SaaS product on your computer right now, open up your developer tools (⌘ + Option + C for chrome) and click on the “network” tab. Click on any button or link in the app, and you should see some events showing up at the bottom. 

🔍 Deeper Look 🔍

The logistics of sending these events gets interesting. Most companies send them client side, e.g. right from your browser. But if you’re using an ad blocker, that blocks any client side JavaScript from hitting external servers, which renders these events useless. For that reason, some companies will set up a proxy server and send these events from their backend; more work, but it avoids the ad blocker problem.

🔍 Deeper Look 🔍

These events get created and fired for pretty much anything you do in the product, and later get assembled into useful insights. But first, the data needs to be stored somewhere.

Getting the data into the warehouse

This is where things start to get complicated. Once the event gets “fired” where does it go? 

These days, the state of the art is to put it straight into your data warehouse. There are a few ways to do that. You can build it yourself – you’d need to build a system that parses the event data and inserts it into Snowflake, BigQuery, etc. via SQL – or, more commonly, use a third party tool like Segment or PostHog to do it for you. 

The end “product” is usually a bunch of tables in your warehouse, one for each type of event. You’ll have a table called `clicked_save_changes`, one called `edited_profile`, and so on and so forth. Sometimes these events will get aggregated into a big events table called, well, `events`. Each one of these tables will have columns for the important data: the time the event happened, which user did it, what page it happened on, etc.

Now we’re getting somewhere!

Transforming the data into something useful

A bunch of raw events aren’t exactly useful for a product manager, even if they can write SQL. How am I supposed to know what my funnel looks like when I’m working with a bunch of random events? The next step in the process is to transform the event data into aggregated, more useful information.

Transforming data today just means writing some SQL that takes these events and filters and aggregates them. For a concrete example, imagine we’re trying to find the most popular “page” in our product, and look at how many users visit each page on a weekly basis. The raw event data looks like this:

But we want the data to look like this:

To do that, we write a bunch of SQL (I will spare you the logistics). State of the art today is to use something like dbt to do this. This is usually the responsibility of the data team, and they will work with business stakeholders like the product team to figure out what these transformations should actually look like.

SQL isn’t necessarily the ideal tool for analyzing event data like this. While we do eventually want to aggregate a lot of this data into rollup tables like the above, sometimes explorations into why users are doing a particular action requires diving into the granular events themselves. Writing custom SQL queries for these over and over again, and analyzing the results tabularly, is a nightmare. In other words, there are still many open questions as to how this type of data should be analyzed most effectively. 

Visualization, BI, and exploration

Once the data is in the format we want it in, we’ll want to get it into a chart of some sort. There are tons of ways to do this, from Metabase to Retool to Hex to homegrown solutions. Even Looker, before it was all but destroyed by acquirer Google, was a tool that teams used for this, among other things. You’ll usually end up with a series of graphs that update regularly and show you the metrics you care about.

The final product isn’t always a static graph, though. More and more, data teams are working on ways to empower non-technical stakeholders to explore data themselves and answer their own questions. 

Tools for product analytics

As you can (hopefully) tell, there’s a lot that goes into that graph showing you how many users your product has, from raw events to transformation and visualization. Though most modern companies will do something like what’s outlined above, there are better and better tools on the market that automate pieces of this process and make them more accessible for non-technical teams. 

There is no substitute for a good data team, and there are always going to be questions that are too deep for SaaS tools to help you answer. That’s why even if you use some of the tools below, chances are you will still have a data team working on the steps outlined above. There are, of course, many many more tools I haven’t mentioned here – leave a comment with any of your favorites.

Segment – instrumentation and routing

Segment helps with multiple parts of the process, from a library for tracking events to routing those events to your warehouse (and other destinations). 

Segment was acquired by Twilio in 2020, so things are a bit up in the air there.

Amplitude – visualization and exploration

Amplitude is a data visualization platform made specifically for events. You can ingest your product events and build charts, dashboards, etc. It’s made mostly for non-technical teams, so you don’t need to write SQL to analyze your data; but as mentioned above, using SQL to analyze event streams isn’t exactly a walk in the park. Personally, I like using Amplitude for questions on the simpler side of things.

The nice thing about Amplitude is it lets you avoid the transformation step for certain questions. E.g. If you want to know what the most popular pages visited in your product are, you can just do that in the Amplitude UI, instead of needing your data team to build a table with that information. As mentioned though, for more complex explorations, this is probably not the tool.

FullStory / HotJar – session recording

FullStory and HotJar are session recording tools: they look at the actions your users take and replay them to you, like you’re watching a video of their screen. Logistically, they’re not actually recording a user’s screen – that would be creepy – they’re instead looking at how the user’s actions impact the HTML of the page, and then rendering what we imagine that probably looked like when the user did it. 

​​Since it’s mostly HTML tagging under the hood, you can filter for things like which page a user visited, what they clicked on, etc.

I have not heard of teams building this functionality on their own, but if you know of any, leave a comment on this post!

PostHog – full stack, everything

PostHog is probably the most unique tool here, because it obviates most of the need for the entire process of building a product analytics practice. Their product extends from event capture all the way to visualization and exploration; they even have functionality around heatmaps and session recording, something you’d normally need to use Fullstory/HotJar for. Their bet is that product analytics should all happen in one place, and that non-technical teams should be empowered to answer almost all of their own questions, with little or no help needed.

PostHog supports all kinds of visualizations – like funnels, user paths, etc. – specifically built for product analytics. They even have support for feature flags, where you release a new feature to a segment of your user base to test how they react to it. Perhaps the most interesting piece of the puzzle: PostHog is open source, which means it’s completely free to run on your own infrastructure (all other products on this list are paid and expensive).

A technical background is a superpower for product managers. You’ll make better decisions, understand trade-offs, make more accurate estimates, and communicate with engineers with more confidence, while also creating more career opportunities for yourself. And now that AI is increasingly infused into products, it’s even more important that product managers understand the basics.

Building a strong foundation of technical skills improves your ability to bridge technical and business domains, uncover implementation issues before they’re in production, and improve your working relationships with your technical counterparts. This quick guide will help get you up to speed and start building your skills with hands-on practice.

Application architecture and APIs

Almost every software product is built with the same three parts:

• a client,

• a server, and

• a database.

  Things can get a lot more complicated as you grow, but these fundamental ideas stay the same.
  The client is your website or app that’s used by the end user.
  The server processes requests from the client.
  The database is your permanent storage of data.

Your client communicates with your server through an application programming interface, or API.

What is an API?

For most software product managers, we’re talking about web APIs. This is the front door to your server that allows other applications to access data or trigger events.

APIs are everywhere—Meta, Etsy, OpenAI, your bank, your doctor, the weather. These kinds of public APIs are meant to be used by developers outside of the company that built them. Anyone who has permission can start making API calls to get data, submit new information, or even interact with AI. 

APIs can also be private. These are intended to be used only within that application or product (versus by third parties).

As a quick example, let’s look at how Airbnb might fetch listings to show you when you search for Toronto.

Breaking this down, we can see that:

1. The client sends a request to listings at the URL api.airbnb.com and specifies the location as Toronto

2. The server gets the listings in Toronto from the database

3. The database returns listings to the server

4. The server returns listings to the client

Wait, so what exactly is the API?

Is the API the server? Kind of. As we said above, the API is the server’s front door—it’s the set of actions that are available to clients. The API (hosted on a server) will either process the request itself or pass it downstream to be processed.

Going back to our Airbnb example, in addition to fetching Listings, you could imagine the API also allows a client to access Users, Bookings, Prices, and more. We commonly call these endpoints or resources.

Let’s take a look at another example: OpenAI. OpenAI supports a wide range of actions via their API. You can create images, transcribe audio, and get chat responses. Here’s what their API endpoints look like:

OpenAI has organized their API into a few groups—images, chat, and audio. Within each, there are a few endpoints that allow you to take different actions.

What are some common patterns with APIs?

Once you understand a few basic patterns with APIs, you’ll be able to breeze through technical documentation in no time.

First, there are three main types of APIs:

• SOAP: This is a legacy design, still used mostly in health care and financial services.

• REST: This is the most popular design pattern; we’ll get into more details below.

• GraphQL: This is a newer design created at Meta, most commonly used in private APIs within your own application. It uses the same request types but in different ways.

As mentioned, REST is by far the most popular design for APIs. It supports many different request types, but there are two we’ll focus on today—GET and POST.

GET requests

GET requests do what you think they would: get stuff. Every time you open a web page, your browser sends a GET request to the server at the URL you’ve entered and the server returns some content to be rendered.

In its simplest form, this content can be an HTML file that your browser displays, or it can be a more complex format that needs to be interpreted, like JSON.

Let’s update our Airbnb example above to specify the request type.

Now we can see that we’re sending a GET request to the /listings endpoint. We also specified that we only want listings in Toronto.

Let’s break the request down into its components:

1. Base URL: The “home” for all API endpoints

2. Endpoint: The specific resource we want to access

3. Query parameter: Additional data passed in with our requests
   
   

   POST requests

   POST requests are used to submit new data to the server. For example, we would use a POST request on Open AI’s /images/generations endpoint to create a new image. 

   We would also include a body with the request, which contains data to be processed by the server. Here’s what that might look like:

As long as we have access, OpenAI will respond to this request with the cutest baby sea otter you’ve ever seen.

Remember that text-to-speech example with an AI-generated voice that we got from OpenAI’s API? Well, that was a POST request.

To generate this, we sent a POST request to https://api.openai.com/v1/audio/speech with the following body:

				
					{
    "model": "tts-1",
    "input": "Did you know subscribers to Lenny's Newsletter get exclusive discounts with partners? Pretty cool! And there's a Slack channel!",
    "voice": "alloy"
  }
				
			

In response to this request, the API sends back an MP3 file with the text-to-speech phrase.

How can I practice?

Want to try a request for yourself? Follow along to generate your own images using DALL-E and OpenAI’s API.

1. Sign up for an API account at https://platform.openai.com/signup 

2. Navigate to API Keys. Verify your account and create your first API key. Make sure you copy it!

3. Using Postman or a web-based API request tool (here’s a free one), make a POST request to https://api.openai.com/v1/images/generations

4. Add the following Headers

   1. Authorization: Bearer [your-api-key]

   2. Content-Type: application/json

5. Add the following body in JSON

				
					{
    "model": "dall-e-3",
    "prompt": "A cute baby sea otter",
    "n": 1,
    "size": "1024x1024"
  }
				
			

6. Send your request!

In today’s fast-evolving tech landscape, the API-first approach is making waves across industries. This isn’t just a trend—it’s a transformative strategy that’s reshaping how companies design, develop, and deliver software solutions. The data speaks for itself: more than 60 API-first companies in 2022 raised $50 million or more in funding, while API-first businesses collectively raised $14 billion. By 2030, experts predict that we’ll see around 1.7 billion active APIs, underlining their central role in the digital economy.

But what does “API-first” actually mean? Why is it so important for businesses today? And how can adopting this approach give organizations a competitive edge? Let’s dive into the technical and business dimensions of API-first development and uncover why it’s becoming a cornerstone of modern software architecture.

What Does API-First Mean?

To understand the API-first approach, it’s essential to grasp the concept of an API itself. An API, or Application Programming Interface, is a set of rules and protocols that allows different software applications to communicate with each other. APIs are like the bridges connecting apps, enabling them to exchange data and work together.

One of the most familiar examples is social media logins. Ever used a website that allows you to log in using Google, Facebook, or Apple? That’s an API at work, seamlessly connecting your social media account to the website you’re accessing.

In the API-first development model, APIs aren’t treated as afterthoughts. Instead, they are prioritized from the very beginning of the development process. This approach is rooted in the idea that APIs should be at the core of how software is designed and built, ensuring that applications are not just functional but also scalable, interoperable, and ready for integration.

The API-First Approach: Technical and Business Perspectives

The API-first methodology benefits both developers and businesses. Let’s explore these advantages from two key viewpoints.

1. Technical Perspective: A Developer’s Dream

From a technical standpoint, the API-first approach is about creating software that’s modular, interconnected, and future-proof. Here’s why it matters:

• Seamless Integration and Connectivity
  APIs act as the glue between different software components. Whether you’re building microservices or integrating third-party platforms, APIs ensure smooth communication and data sharing. This modularity makes it easier to assemble complex systems from smaller, reusable components.

• Cross-Platform Compatibility
  APIs are designed to work seamlessly across devices and platforms, ensuring that software can cater to diverse user bases. Whether it’s a mobile app, a web application, or even IoT devices, APIs enable consistent experiences.

• Scalability and Flexibility
  As businesses grow, so do their software requirements. APIs enable developers to scale systems by adding or updating components without disrupting the entire ecosystem. This adaptability is crucial in accommodating larger user bases and evolving business needs.

• Enhanced User Experiences Through Data Integration
  APIs can pull data and features from various sources, creating applications that deliver richer, more dynamic user experiences. For example, a travel app can combine flight data, hotel availability, and weather forecasts—all through APIs.

2. Business Perspective: Unlocking New Opportunities

The API-first approach isn’t just a technical strategy—it’s a powerful business enabler. Companies like Stripe, Twilio, and AssemblyAI have demonstrated how API-first can drive growth and innovation. Here’s how it works:

• Monetization Opportunities
  APIs can be a revenue stream in themselves. Companies can charge for API usage, offer premium features, or establish partnerships with other businesses that rely on their APIs. For instance, Stripe’s APIs power payment systems globally, generating significant revenue.

• Expanding Market Reach
  APIs enable third-party developers to build applications and services on top of your platform. This creates a developer ecosystem and helps your business expand its market presence. Think about how platforms like Twilio have become indispensable to developers building communication tools.

• Faster Time-to-Market
  With APIs, companies can reuse existing software components, reducing development time. This allows startups and businesses to bring products to market faster, staying ahead of competitors.

• Competitive Edge Through Integration
  In today’s software landscape, customers expect tools that integrate seamlessly with others. API-first companies are better positioned to meet this demand, providing superior integration options that attract and retain clients.

• Agility and Innovation
  By prioritizing APIs, companies can quickly adapt to changing market conditions. APIs enable businesses to iterate and innovate faster, ensuring they stay relevant in dynamic industries.

Key industries for API-first development

While the API-first approach can bring value across various domains, here are sectors where this approach is especially beneficial. 

Healthcare

In healthcare, APIs are crucial for bringing together multiple players and ensuring the data is shared between them in an accessible and secure manner.

• Healthcare organizations must securely transmit electronic health records (EHRs) and patient information among various providers, insurers, and systems. API-first integration ensures the seamless and secure exchange of medical data, improving patient care coordination. 

• More than that, APIs enable the integrating of disparate healthcare systems, such as electronic medical record (EMR) systems, telehealth platforms, and diagnostic equipment. APIs also empower developers to create innovative healthcare applications, including mobile health (mHealth) apps, remote patient monitoring solutions, and telemedicine platforms. These applications can transform the way patients access care and manage their health. 

For example, SmartHMS, an EHR provider, offers APIs that allow healthcare organizations to integrate patient data seamlessly. This enables healthcare providers to access patient records, exchange information securely, and improve care coordination.

Banking and finance

The financial industry relies heavily on data and transactions, making the API-first approach beneficial for improving services and operations.

• API-first strategies are at the heart of open banking initiatives, enabling financial institutions to expose their services and data to third-party developers securely. This encourages the creation of new financial products, and enhances customer experiences.

• APIs facilitate real-time access to financial data, enabling customers to view account balances, conduct transactions, and receive alerts instantly. This real-time capability is essential for online banking and mobile banking applications.

• APIs can be used for fraud detection and prevention services, enhancing security in financial transactions, and enabling rapid detection of suspicious activities and help mitigate fraud risks.

The most notable example here is, of course, Razorpay. Razorpay's API-first approach revolutionized online payments. Its APIs enable businesses to integrate payment processing into their websites and apps seamlessly, simplifying the checkout process and enhancing the user experience.

Travel and hospitality

This industry relies on seamless customer experiences, making the API-first approach invaluable for improving services and expanding offerings.

• APIs allow travel companies to easily connect users with airlines, hotels, car rental services, and other providers, streamlining the booking and reservation process, and offering customers a one-stop-shop for travel arrangements.

• By leveraging APIs, travel companies can access customer data and preferences, enabling them to offer personalized recommendations, travel itineraries, and loyalty rewards. This personalization enhances the overall travel experience.

• APIs enable real-time access to pricing and availability information. Travel companies can adjust pricing based on demand and market conditions, maximizing their revenue and competitiveness.

Amadeus provides APIs for travel booking and reservation systems. Airlines, hotels, and travel agencies can integrate their APIs to offer comprehensive travel booking and management services, improving the traveler's experience.

API-first approach and UX design

When you think about launching a SaaS business, you should always keep user experience (UX) at the forefront. And going for an API-first design is no exception. 

The API-first design and development allows for a more personalized and seamless user experience. For example, by combining API functionality with thoughtful UX design you can better address users’ needs and preferences. APIs provide the data and functionalities necessary for a feature-rich application, while UX design ensures that these features are intuitive and user-friendly. 

UX designers can also leverage APIs to streamline user journeys. For example, integrating geolocation APIs can enhance the user experience by providing location-based services, such as finding nearby stores or restaurants, letting users achieve their goals much easier.

Finally, the collaboration between API development teams and UX designers promotes an iterative approach to product development. Developers and designers rely on user feedback, as well as on each other's insights to continuously improve the product.

1. Requests & Responses

APIs work through requests and responses—much like a conversation between two systems. You make a request, and the API returns a response, typically in JSON or XML format.

AI Example: When you use an AI chatbot, it sends a request to an API to fetch user details, and the response includes personalized recommendations based on past behavior.

Why It’s Important: Understanding this concept helps you know how data moves between systems, aiding in integrations with other platforms.

2. REST (Representational State Transfer)

REST is an architectural style that makes APIs more scalable and easier to use. It relies on standard web protocols like HTTP for communication.

AI Example: Popular AI services like OpenAI or Google Cloud’s AI services use REST APIs to process text or image inputs and return predictions or classifications.

Why It’s Important: RESTful APIs make it easier to manage and scale services, especially when your AI product grows in user base.

3. HTTP Methods

APIs use specific HTTP methods like:

• GET: Retrieve data

• POST: Submit new data

• PUT: Update existing data

• DELETE: Remove data

AI Example: An AI-powered app might use a POST method to submit new images for object recognition, while a GET request retrieves the result of the analysis.

Why It’s Important: Knowing these methods helps you understand what actions your API is performing, whether it's retrieving user info or updating a database.

4. Endpoints

Endpoints are specific URLs where API resources or services are accessible. Think of them as access points for different functions of an API.

AI Example: In an AI image recognition system, you might have an endpoint like /api/v1/images/upload to upload photos for analysis.

Why It’s Important: Well-structured endpoints make APIs more intuitive to use, which can speed up product development.

5. API Documentation

API documentation is a guide that explains how to use the API, listing available endpoints, data formats, and authentication methods.

AI Example: When integrating an AI text-to-speech API, the documentation will tell you what file formats are accepted (e.g., WAV or MP3) and how to structure your requests.

Why It’s Important: Clear documentation ensures smooth integration, helping your team understand how to use the API without needing to constantly ask the provider for help.

6. API Calls

An API call is the request made to an API to retrieve or send data. It involves specifying the endpoint, method (like GET or POST), and any necessary parameters or payloads.

AI Example: An e-commerce platform might make an API call to an AI-powered recommendation engine to get product suggestions for users based on their browsing history.

Why It’s Important: Every interaction between your product and an external service is an API call, so understanding this concept is crucial for seamless integrations.

7. Payloads

The payload refers to the data that you send to or receive from an API. This could be in the form of user data, settings, or a file.

AI Example: When sending text to an AI sentiment analysis API, the payload contains the actual text. The response payload might contain a sentiment score (positive, neutral, or negative).

Why It’s Important: Managing payloads correctly ensures that the right data is being transferred and processed, which is especially important for AI products that rely on large datasets.

8. Response Codes

Response codes are numeric codes that indicate the status of an API request. Common codes include:

• 200 OK: Request succeeded

• 404 Not Found: The endpoint doesn’t exist

• 500 Internal Server Error: Something went wrong on the server

AI Example: In a machine learning model deployment, a 200 response code indicates that the model successfully processed the input, while a 400 (Bad Request) could indicate an issue with the data format.

Why It’s Important: These codes give you insight into what’s happening under the hood, allowing you to troubleshoot issues more efficiently.

9. Headers

Headers are part of an API request or response that provide additional information, like content type or authorization tokens.

AI Example: When interacting with an AI service like AWS Rekognition, you’ll often need to include an authorization header with your API key to access the service.

Why It’s Important: Properly using headers is crucial for authentication and managing data formats, ensuring secure and efficient communication.

10. Authentication

Authentication ensures that only authorized users can access an API. Common methods include API keys, OAuth tokens, and JWT (JSON Web Tokens).

AI Example: Many AI platforms, like IBM Watson, require OAuth-based authentication to ensure only authorized users can make API calls to train or access models.

Why It’s Important: Security is paramount when building products that rely on sensitive data, especially in AI where proprietary algorithms and data are key assets.

Why These Terms Matter for Product Managers

Understanding these API terms enables you to:

• Collaborate effectively with technical teams: You’ll be able to speak the language of developers and engineers, reducing the communication gap.

• Make better product decisions: Knowing how APIs work helps you evaluate third-party services and decide which are best for your product.

• Optimize API integration: By understanding API documentation, you can ensure smooth and efficient integrations, saving time and resources during development.

Conclusion

APIs are essential for building scalable, AI-driven products. By familiarizing yourself with these key API terms, you can better manage your product's technical aspects, ensuring successful integrations and smoother collaboration with your development team.

Stay updated with new API trends and practices to keep your product competitive in a rapidly evolving digital landscape!