
## The Big Idea

When providing accurate context is make-or-break for your workflow, everyone needs to become a better writer.

We convinced ourselves that moving fast was the same as being smart. Shipping first. Reacting faster than anyone else. What we did not notice is that this trained an entire generation of builders to think only in the moment, never on the record.

That mindset worked when everything stayed in your own head. It breaks completely when your collaborators are machines. And it is the same thing your teammates have quietly been begging you to fix for years.

### Why this suddenly matters

- With LLMs and agents, context is not just helpful, it is _everything_.
- Developers are historically allergic to context. We want to skip the ticket and jump to the code.
- That habit is now actively expensive.
- Clear writing is fast implementation. Fuzzy writing creates hallucinations, bugs, and rework, and wastes time.
- If you get good at writing for agents, you automatically get better at writing for humans. Your teammates have been asking for this for years anyway - they're gonna love you for it.



## A simple framework for writing to agents (and humans, and future-you)

1. State the problem or goal. What is happening? What outcome matters?
2. Provide history. What have you already tried, seen, or ruled out?
3. List constraints. What absolutely must not break?
4. Point to the code. If you were starting, where would you look first? Any past examples or similar patterns?

Go back and re-read your instructions, and imagine this is the first time you're hearing about the bug or feature you're describing. Is there enough information for you to get the job done?

Iterate as needed. That's really all there is to it.

## A real example

I was getting spam signups to the Tiny Improvements mailing list. Dozens a day. Instead of stewing on it internally like I used to, I wrote up this description from my phone while on a plane:

```
I have been getting spam signups on my newsletter form for the past couple weeks - can you read through `SubscriptionForm.tsx` and help me come up with a plan to deter spam signups?

Here are some sample subscriptions from the past few weeks that are all spam:
(list of ~50 spam signups, including the addresses, first and last names used)

Based on what I'm seeing here, I think we can probably do a few things to deter spam signups:
1. make sure the honeypot field on the form is working as expected
2. Automatically reject signups where the first name is mixed case has >3 capital letters: i.e. XavcASkiox
3. Automatically reject signups where the first name has >4 consecutive consonants: i.e. mtpDQVeZaqb

For signups that _fail_ any of our spam checks, we should make it look like the subscription was a success, but don't actually subscribe the email address, and instead log the spam signup attempt to Posthog analytics for further analysis.
```

That 90 seconds of writing turned into a working fix with minimal edits, and thankfully, spam fell off a cliff.

And bonus: writing it down cleared background mental noise I did not realize I was carrying.

## Conclusion

If context is now make-or-break, writing is no longer optional - it's the difference between wasting your time and speeding up your workflow.

Write down what you're thinking about. It frees your brain, helps your team, and makes agents actually useful.

_And if you're the one sending automated spam signups to my newsletter: c'mon, dude._

---

### For more Context on Context

- [Matt Pocock's video on LLM Context](https://youtu.be/-uW5-TaVXu4) is a superb explainer on the topic, with a really helpful series of diagrams to help you visualize the concept.
- Michael Reeves shows what happens when you [corrupt the back-context](https://www.youtube.com/shorts/WP5_XJY_P0Q) of an LLM conversation. It's chaotic - and convincing.


The next great engineering skill is not coding

A simple framework for writing to agents (and humans, and future-you)


## A Quick Win with text-wrap: balance

Here's a ludicrously simple tip to make your website more readable: use the `text-wrap: balance` CSS property. It's one line of CSS, but it makes a noticeable difference in how text appears, especially for headings.

### Why Use text-wrap: balance?

When text breaks across multiple lines, it often creates uneven line lengths, which can feel awkward to read. The `text-wrap: balance` property ensures that text lines are balanced, creating a more visually pleasing and readable layout.

Take a look at these two versions of a heading:

**Before:**

### What is text-wrap: balance?

`text-wrap: balance` is a CSS property designed to improve the visual balance of text that spans multiple lines. By default, when a block of text wraps onto a new line, the lengths of those lines can vary significantly, creating uneven and sometimes awkward breaks.

With `text-wrap: balance`, the browser automatically adjusts the wrapping of text so that the lines are as close to equal length as possible. This makes the text easier to read, and looks great - particularly for headings, short paragraphs, and blockquotes.

### Key Points:

- **Purpose**: Ensures evenly distributed line lengths in multi-line text.
- **Use Case**: Best suited for headings, short paragraphs, or any text that benefits from visual balance.
- **Compatibility**: Works on most modern browsers and gracefully falls back to default text wrapping behavior where unsupported.

By enhancing the balance of text, this property helps to create a polished and professional design with minimal effort.

## See text-wrap: balance in action

Here's a quick side by side example of a heading before and after using `text-wrap: balance`:

<heading className="text-3xl font-bold text-purple-700">
  This is a heading that doesn't have the rule applied; text will break onto a
  new line unevenly
</heading>

<br />

And by contrast -

<br />

<heading className="text-3xl font-bold text-balance text-purple-700">
  This is a heading that will feel nicely balanced if it spills onto a new line
</heading>

(Note: you may need to resize your browser window to see the difference)

## How to use it: CSS

One little line of CSS does the trick:

```css
h1,
h2,
h3,
h4,
h5,
h6 {
  text-wrap: balance;
}
```

## How to use it: Tailwind

Recent releases of Tailwind have built-in support for `text-wrap: balance`. You can start using it with the `text-wrap-balance` class.

```
<h1 class="text-wrap-balance">
  This is a heading taht will feel nicely balanced if it spills onto a new line
</h1>
```

### Browser Support

According to Can I Use, `text-wrap: balance` is supported by approximately 90% of browsers. While there are exceptions (e.g., IE11, Opera Mini, and some niche mobile browsers), the fallback is harmless: the text will wrap as it normally would without the property.

### Why It Matters

Legibility and readability are critical for user experience. There's even a psychological principle-the [Aesthetic Usability Effect](https://mikebifulco.com/newsletter/aesthetic-usability-effect) - that suggests people perceive more attractive designs as easier to use. Balancing your text lines is a subtle but effective way to make your site look polished and professional.

## Try It Today

Improving the look and feel of your site doesn't have to involve a complete overhaul. Small changes like text-wrap: balance can make a big difference. Add it to your site, and give your visitors a better reading experience-one balanced line at a time.

<details>
  <summary className="text-lg font-bold inline ml-4 cursor-pointer text-pink-600">
      Video Transcript
  </summary>

[00:00:00] **Mike Bifulco:** Hey, what's up friends. This is Mike from tiny improvements. I have a ludicrously simple tip for you.

[00:00:03] That's going to make your sites look better with one line of CSS. And I cannot believe how easy this is, which is why I'm making a video about it.

Side by side Demo: Single post

[00:00:10] **Mike Bifulco:** What we're looking at here is my website. I've made a change. That is exactly one line of CSS between this and the version you're about to see.

[00:00:16] And I think the change is tremendous. Take a look at this.

[00:00:21] Again, version one. Version two.

[00:00:23] Version one. Version two. Version one. How to set up self healing, URLs, and next JS for better. SEO. Version two, how to set up self healing, URLs, and next JS for better SEO. I think that's a little more readable. Don't you?

[00:00:38] Maybe he doesn't pronounce enough here. Let's look at a more detailed example.

Side by Side example: Lots of posts

[00:00:41] **Mike Bifulco:** this is my newsletter page, where I list off many newsletters that I've written in the past.

Subscribe to 💌 Tiny Improvements

[00:00:45] **Mike Bifulco:** If you're watching this and you don't subscribe already I would love it if you did. I write a weekly newsletter called tiny improvements with product tips for startup founders, product builders, and indie hackers, based on my experience, building startups in the past and currently building my startup Craftwork, which is a Y [00:01:00] Combinator backed startup.

Looking at the change

[00:01:00] **Mike Bifulco:** now let's take a look at this change here.

[00:01:02] It's much more pronounced when you see it multiple times.

[00:01:04] So, what we're looking at is CSS change that brings balance to text when it breaks across lines. And that is almost exactly what the change is. It is CSS, text balance.

[00:01:14] If I pull this up on the Chrome for developers site, you can see from my pal, Adam Argyle, a super straight forward tutorial on what this thing looks like.

[00:01:24] And the change is pretty easy to see when it's animated like this. Although it's a little hard to describe what's happening is the balanced version is trying to make the text break into lines that are roughly evenly sized.

[00:01:33] So it has a lot more visual balance. It's a little easier to parse because your eyes and jumping from one line to the next isn't quite as far of a distance on the balanced version versus the unbalanced version. It also works with Arabic text, right to left left to right. It doesn't matter.

[00:01:46] There are some limitations here.

[00:01:48] This balance rule works on up to six lines, which means that if you apply it to everything, it won't work for long paragraphs, for example. But the nature of this rule really applies to headings. This is more or less the way that this should be applied [00:02:00] and I'll show you how I did it on my site.

[00:02:01] Next.

Implementation details in Next.js with Tailwind

[00:02:01] **Mike Bifulco:** This is what you have to do to make this thing happen on your site, where you're implementing headings. You need to add a text balance rule to it. This has done with tailwind. So it's applying the text wrap, colon balance CSS property to my headings. And this component happens to apply to all the headings on my site. So it was super easy to do, but if you had to do it somewhere else, it would look something like this. Const balanced, heading. Equals. Return H one style equals there you go.

[00:02:31] Tax rate balance. This would do it. If you need to do it, one-off for some reason, but you could also apply this to block quotes. Like the Google article explained. So const balanced block quote equals tax wrap balance, or in tailwind land, you could do. Class name equals text balance, not work just as well.

[00:02:51] Matter of fact, let's go do this for the block quote on my site too.

[00:02:53] I have this. Lovely block quote, component that I use to render block quotes on my site. And I think it's a reasonable thing [00:03:00] to add text balance here as well. So we're going to do that and that will improve the aesthetics of my site. That's super nice. It's very subtle. People may never notice that this happens, but it should make it so that when your title's going to a second line, the look that much nicer, pretty cool, right? I thought so.

## Can I use it yet though?

[00:03:13] **Mike Bifulco:** Last thing I know what you're thinking. Is this something that I can actually use yet? Well, the good news is pretty much, yes. According to can I use this is used by almost 90% of browsers on the internet, which is fantastic. My apologies to people who use IE 11 opera mini UC browser for Android QQ, browser, Baidu browser, or Caio S browser.

[00:03:32] You're going to have to wait a little bit longer to see nice titles on my site, but in the meanwhile, the fallbacks. All back scenario for this is that your text will rep as it used to, if you're using this rule. So I consider this relatively safe to use.

[00:03:44] It's also already supported by tailwind, which is fantastic.

[00:03:47] I think you should go add this to your site and give it a shot. Legibility and readability are really important for user experience. And there's this thing called the aesthetic usability effect that the better looking your site is the better looking your product is.

[00:03:58] The more people will [00:04:00] inherently like it. It's a scientifically proven thing that, the better looking something is the more people put value in it.

[00:04:05] So why not? I don't want line change like this.

[00:04:07] That gets you some real wins for free.

[00:04:09] All right.

[00:04:09] That's a quick tip. I hope it's helpful for you until next time.

[00:04:12] My name is Mike Bifulco. This is a tiny improvements. Give them hell out there.

[00:04:15] ​

</details>


CSS Tips - text-wrap:balance will make your sites feel better


## My writing workflow

I have worked diligently to develop a writing workflow that keeps me productive and focused on writing, rather than getting hung up on the tools I use to write.

### A little about me

I'm Mike Bifulco, a software engineer and designer. I work as CTO and Cofounder of [Craftwork](https://craftwork.com) (Y Combinator S23), and I publish a weekly newsletter called Tiny Improvements to a focused audience of startup founders, product builders, and indiehackers.

I have worked in developer relations and content creation since about 2017, and have been publishing content on [mikebifulco.com](https://mikebifulco.com) since 2019.

Along the way, I've learned a lot about how to write and publish content, and I've developed a workflow that keeps me productive and focused on writing, rather than getting hung up on the tools I use to write.

My site is built with Next.js, and I use Markdown (and something called MDX) to write posts.

Sharing my work online has created endless connections and opportunities for me; if I had never hit the `publish` button on my work, I would not be where I am today.

### Why workflow matters

Time and time again, I've seen aspiring writers, devrel-ists, and content creators waste weeks, months, and sometimes years trying to get everything _just right_ before they share their work with the world. That is a waste of time, and it's a shame.

Regardless of who you are, the tools you use, and what you're writing about, landing on a workflow that works for you is the first step to getting started.

If the work you're creating is good, you've got a great foundation to stand on. Sharing your expertise and knowledge with your community starts there - the rest is just details.

## How I publish content on my Next.js site with Markdown and MDX

For all content on my site, my authoring workflow looks like this:

### Write in Markdown

<Image
  publicId="posts/content-creation-workflow-my-writing-process/writing-screenshot"
  alt="A screenshot of my IDE with a Markdown file open"
  caption={
    <span>
      writing new posts looks a whole lot like{' '}
      <a href="https://mikebifulco.com/posts/self-healing-urls-nextjs-seo">
        this
      </a>
      .
    </span>
  }
/>

1. I write in [Markdown](https://daringfireball.net/projects/markdown/). This makes it easy to write without distraction, and keeps my writing process focused on _what_ I should be saying, rather than how it looks.
1. I create a file for each new post in [the source code for my site](https://github.com/mikebifulco/blog) under the `src/data/newsletters` directory, as a single `.mdx` file with a small amount of frontmatter (title, desctiption, tags, slug, and a cover image).

### Every article needs a cover image

<Image
  publicId="posts/content-creation-workflow-my-writing-process/figma"
  alt="A screenshot of a Figma file with a cover image for an article"
  caption={
    <span>
      <a href="https://mikebifulco.com/newsletter/open-sourcing-my-design-system">
        I use Figma
      </a>{' '}
      to create cover images for all of my articles.
    </span>
  }
/>

1. For every article, I create a `1200 x 630px` cover image in [Figma](https://mikebifulco.com/newsletter/open-sourcing-my-design-system). This is the image you'll see on social media sites when a link to my articles is shared. Any other images included in the body of a post are created the same way, and are generally the same size.
1. I use [Squoosh](https://squoosh.app) to compress and optimize any included images, and then upload them to [Cloudinary](https://cloudinary.com) as `.webp` files.

### Commit, merge, and publish

<Image
  publicId="posts/content-creation-workflow-my-writing-process/pull-request"
  alt="A screenshot of a pull request for this very post in GitHub"
  caption="This is what a pull request looks like for this very post."
/>

1. When I'm done writing and ready for a newsletter to go live on my site, I commit the changes to a new branch, and open a pull request _to myself_ to merge the changes into the `main` branch. This generates a preview build of the new page on Vercel, where I can check out the changes and make sure everything looks good before merging.
1. After the PR is merged, Vercel automatically builds the site and deploys the changes, and the article is live in a few minutes.

## After I publish: sharing my work

**The most critical part of this process is sharing my work with the world.**

Once a post goes live, I run off to social media and start _sharing my work_ with the world - on Bluesky, on Threads, on LinkedIn, and on Reddit. The ins and outs of that process is a whole other article, but I'll say this: it's just as important as _actually publishing something_.

## Go hone your process

Here's the thing: this is _my_ workflow - it works great for me, because I built it around my own preferences and needs. As you're designing your workflow for publishing, take advantage of the things your brain likes.

Is it easier for you to write in Notion? Great! Start there. Is Markdown a pain in the butt? Skip it and use something that you like better. There's no right anwers here - the only thing that matters is that you actually get over the hump and _publish something_.

Once you've published something, you can iterate from there.

Good luck!


Content creation workflow: my writing process

How I publish content on my Next.js site with Markdown and MDX


## Transitioning from next-sitemap to Next.js's Built-in Sitemap.xml and Robots.txt APIs

In this post, I'll walk you through the process of migrating from the next-sitemap npm package to using Next.js's built-in API for generating sitemap.xml and robots.txt files. This transition not only simplifies our codebase but also leverages the capabilities of Next.js to enhance our SEO strategy.

### Why Migrate?

The [next-sitemap npm package](https://github.com/iamvishnusankar/next-sitemap) was a great tool for generating sitemaps, but as Next.js has evolved, it has introduced built-in support for sitemap and robots file generation. This change allows for a more streamlined approach, reducing dependencies and improving maintainability.

Admittedly, I have also been a bit of a laggard in updating my site to the new app directory - this is actually the _very first_ use of App Router on this site!

## What is a sitemap?

A sitemap is a file that lists all the URLs on your site, helping search engines discover and index your content. It provides a roadmap for search engine crawlers to navigate your site and understand its structure. They're also used by other services like Google Search Console to understand your site's structure. If you haven't used Google Search Console, it's an invaluable resource for measuring your site's search engine performance.

I've written at length about the value of SEO, including the [SEO tools I used to grow my sites to 20k+ visitors/month](https://mikebifulco.com/posts/seo-tools-for-new-projects).

## What is a robots.txt file?

The robots.txt file is a configuration file used to instruct web crawlers (like Googlebot) on how to crawl your site. It allows you to specify which pages should be indexed and which should be excluded, helping control the visibility of your content to search engines. In this case, we're using robots.txt to stop search engines from indexing URLs that we don't want to show up in search results, like Next.js API routes, and other pages that are generated dynamically.

## Removing the next-sitemap dependency

The first step is to remove the next-sitemap dependency from your project:

1. remove `next-sitemap` from your package.json file - make sure to check both `"scripts"` and `"devDependencies"`, and remove it from both places.
2. run `npm install` (or `yarn install` or `pnpm install`) to update your lockfile
3. delete your `next-sitemap.config.js` file if it exists
4. **Important:** If you have `sitemap.xml`, `sitemap-0.xml`, etc. files in the `/public` directory of your repo, delete any that exist. Next won't generate a sitemap file if these are already there - and it can be tricky to debug!

Side note: [`next-sitemap`](https://www.npmjs.com/package/next-sitemap) was a fantastic tool, and served me for quite a long time. I'm super thankful to the maintainers for their work on the package - it filled an important need for a _very_ long time!

## Generating a sitemap.xml file with Next.js

Next.js has a great built-in API for generating sitemaps and robots files. We'll use this API to generate our sitemap.xml and robots.txt files. The API is simple to use and allows for customization of the sitemap and robots file generation process.

There are a couple choices for [creating a sitemap with Next.js](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/sitemap#generating-a-sitemap-using-code-js-ts):

1. **Write and maintain the XML for your sitemap yourself** - this may be okay for really small sites, but you will need to add an entry to this file for _every_ page you want search engines to know about. This quickly becomes cumbersome for sites with dynamic content, or even just a lot of pages.
2. **Use Next.js's built-in API to generate the sitemap.xml file** - I am going to opt for the built-in API because my site has hundreds of pages published, and I don't want to have to maintain a sitemap file with an entry for every single page.

### Configuring the sitemap.xml file

Create a new file in the App directory called `sitemap.ts`:

```tsx
import type { MetadataRoute } from 'next';

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  return [
    // TODO: one entry for every page you want search engines to know about
  ];
}
```

This file is where you will add an entry for every page you want search engines to know about. The goal is to return an array of objects of the following type:

```tsx
type SitemapFile = Array<{
  url: string;
  lastModified?: string | Date;
  changeFrequency?:
    | 'always'
    | 'hourly'
    | 'daily'
    | 'weekly'
    | 'monthly'
    | 'yearly'
    | 'never';
  priority?: number;
  alternates?: {
    languages?: Languages<string>;
  };
}>;
```

A couple things to note here, from the [sitemap spec](https://www.sitemaps.org/protocol.html#xmlTagDefinitions):

1. The `url` property is required and should be the full URL to the page you want to include in the sitemap.
2. The `changeFrequency` property is optional and can be one of the following values: `always`, `hourly`, `daily`, `weekly`, `monthly`, `yearly`, or `never`. This is a _hint_ to search engines about how often the page is likely to change. Google and other search engines may choose to use different values than you specify if it becomes clear that your page changes at a different rate than you specify.
3. The `priority` property is "The priority of this URL relative to other URLs on your site." It is an optional property and can be any number between 0 and 1. If you don't include it, it will default to 0.5 for every page. I recommend setting this to 1 for your homepage, and cascading it down to 0.5 for other pages, depending on their relative importance to your site. It may not actually influence search engine behavior - and it's not a guarantee that a page will rank higher just because it has a higher priority - but it is a useful hint for search engines.
4. `alternates.languages` is a feature of Next.js that allows you to specify alternate versions of your page in different languages. This is useful if you have a blog post that is published in multiple languages, and you want to include all of the versions in your sitemap. I only publish in English (_en este momento!_), so I haven't tested this yet, and it isn't included in this tutorial.

### Populating your sitemap.ts file

#### Static pages

For static pages on my site, I opted for a semi-manual approach to adding them to the sitemap. I created an array of strings that represent the routes I want to include in the sitemap, and then mapped over that array to create the sitemap entries.

```tsx
// Define your static routes
const routes: string[] = [
  '', // home page
  '/about',
  '/integrity',
  '/newsletter',
  '/podcast',
  '/posts',
  '/tags',
  '/work',
  '/shop',
];

// Create sitemap entries for static routes
const staticRoutesSitemap = routes.map((route) => ({
  url: `${baseUrl}${route}`,
  lastModified: new Date(),
  changeFrequency: 'weekly' as const,
  priority: route === '' ? 1 : 0.8,
}));
```

When run, this will generate an array of sitemap entries for each of the routes in the `routes` array - note that the homepage has a priority of 1, and all other pages have a priority of 0.8.

#### Dynamic pages

For pages that are generated dynamically, or hosted by your CMS, the process isn't too different. You will need to fetch the URL of each page you want to include in the sitemap, and then create a sitemap entry for each one. In my case, I have 3 different sources of dynamic content:

1. Newsletters: these are dispatches of my newsletter [💌 Tiny Improvements](https://mikebifulco.com/newsletter), which I send out every week.
2. Blog posts: these are the blog posts I publish on this site (you're reading one right now!).
3. Tags: These are metadata tags I use to categorize my blog posts and newsletters issues. I generate a list of all tags on the site, and then map over that list to create a sitemap entry for each one.

I write all my site content in [`MDX`](https://mdxjs.com/docs/what-is-mdx/), and save them as files in my site's repo under the `src/data/` directory.

I used a utility function to grab all of the newsletter issues, blog posts, and tags from the `src/data/` directory, and then mapped over each one to create a sitemap entry. For newsletters, that looks like this:

```tsx
// this is a helper function used across my site.
// replace with whatever is needed to fetch from the CMS you use
const newsletters = await getAllNewsletters();

// create a sitemap entry for each newsletter
const newslettersSitemap = newsletters.map((newsletter) => ({
  url: `${baseUrl}/newsletter/${newsletter.slug}`,
  lastModified: new Date(newsletter.frontmatter.date),
  changeFrequency: 'weekly' as const,
  priority: 0.8,
}));
```

The same pattern is used for blog posts and tags.

## Sitemap.ts: The complete file

```tsx
import type { MetadataRoute } from 'next';

import { getAllPosts } from '@lib/blog';
import { getAllNewsletters } from '@lib/newsletters';
import { getAllTags } from '@lib/tags';
import { BASE_SITE_URL } from '@/config';

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const baseUrl = BASE_SITE_URL;

  const newsletters = await getAllNewsletters();
  const newslettersSitemap = newsletters.map((newsletter) => ({
    url: `${baseUrl}/newsletter/${newsletter.slug}`,
    lastModified: new Date(newsletter.frontmatter.date),
    changeFrequency: 'weekly' as const,
    priority: 0.8,
  }));

  const posts = await getAllPosts();
  const postsSitemap = posts.map((post) => ({
    url: `${baseUrl}/blog/${post.slug}`,
    lastModified: new Date(post.frontmatter.date),
    changeFrequency: 'weekly' as const,
    priority: 0.8,
  }));

  const allTags = await getAllTags();
  const tagsSitemap = allTags.map((tag) => ({
    url: `${baseUrl}/tags/${tag}`,
    lastModified: new Date(),
    changeFrequency: 'weekly' as const,
    priority: 0.5,
  }));

  // Define your static routes
  const routes: string[] = [
    '', // home page
    '/about',
    '/integrity',
    '/newsletter',
    '/podcast',
    '/posts',
    '/tags',
    '/work',
    '/shop',
  ];

  // Create sitemap entries for static routes
  const staticRoutesSitemap = routes.map((route) => ({
    url: `${baseUrl}${route}`,
    lastModified: new Date(),
    changeFrequency: 'weekly' as const,
    priority: route === '' ? 1 : 0.8,
  }));

  // combine all the sitemap entries into a single array
  return [
    ...staticRoutesSitemap,
    ...newslettersSitemap,
    ...postsSitemap,
    ...tagsSitemap,
  ];
}
```

### Testing your sitemap

Fire up `next dev` and navigate to `/sitemap.xml` - you should see your sitemap! Make double and triple sure that your sitemap has an entry for every single page you want search engines to know about. You may also want to check that it _updates_ correctly by adding a new page and checking that it appears in the sitemap.

<Aside type="warning">

If it doesn't seem like your sitemap is updating correctly, **remember to check your `/public` directory for sitemap.xml files** that may have been generated by `next-sitemap`, and delete them!

</Aside>

## Creating a robots.txt file

Next up, robots.txt. This is a _very_ similar process - we will follow the [docs for robots.txt](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/robots) from Next.js.

This is a deceptively important file for your website's SEO. It is a really good idea to familiarize yourself with the [robots.txt file format](https://www.robotstxt.org/robotstxt.html), and to understand the meaning of each rule you set.

### Disallow everything for non-production environments

**🚨 Important: ** If you are running a non-production environment, you will need to add a check to your robots.ts file to disallow all requests. This is because non-production environments often have URLs that you don't want search engines to index - it would be bad news if Google decided that a deploy preview of your site should rank higher than your production site!

### Robots.ts: The complete file

Much like static pages for the sitemap, you will need to add an entry for every path you don't want search engines to index.
In my case, I have a reverse proxy in front of my site that handles analytics and logging with PostHog, and I don't want search engines to index those URLs.
You'll see that handled below with `noIndexPaths`.

```tsx
import type { MetadataRoute } from 'next';

import { BASE_SITE_URL } from '@/config';

const noIndexPaths = [
  '/ingest', // posthog's reverse proxy
  '/ingest/*', // posthog's reverse proxy
];

export default function robots(): MetadataRoute.Robots {
  // 🚨 IMPORTANT: if this is not a production environment, disallow all requests
  if (
    // Vercel-specific environment variable. Please check the docs for your hosting provider!
    env.VERCEL_ENV !== 'production' ||
    // for a generic node environment
    process.env.NODE_ENV !== 'production'
  ) {
    return {
      rules: [
        {
          userAgent: '*',
          disallow: '*',
        },
      ],
    };
  }

  return {
    rules: [
      {
        userAgent: '*',
        disallow: '/api/', // Next.js API routes
      },
      {
        userAgent: '*',
        disallow: '/_next/', // Next.js build output
      },
      {
        userAgent: '*',
        disallow: '/public/', // static files like css, images, fonts. This one's up to you!
      },
      ...noIndexPaths.map((path) => ({
        userAgent: '*',
        disallow: path,
      })),
    ],
    sitemap: `${BASE_SITE_URL}/sitemap.xml`,
  };
}
```

A few important callouts here: `BASE_SITE_URL` is a variable I set in my site's config, and it is the full URL to the root of my site (`https://mikebifulco.com`). You will need to replace it with the actual URL of your site.

## Deployment checklist

Now that everything's done, make absolutely sure you've got everything configured correctly. Here's a quick checklist I used to verify that my new sitemap and robots.txt were set up correctly:

```md
**Before going live**, verify that:

- robots.txt
  - [ ] visit `/robots.txt` and verify that it exists
  - [ ] check that there is an entry for your sitemap that contains a full URL, including the protocol (`https://`)
  - [ ] read through the disallow rules to make sure they make sense for your site. If you're using next.js, this will include `/_next/` at a minimum, and possibly `api/` and `public/` if don't want those directories indexed.
- sitemap.xml
  - [ ] visit `/sitemap.xml` and verify that it exists
  - check that it contains the correct entries for all:
    - [ ] static pages
    - [ ] dynamic pages (newsletters, blog posts, tags)

**While testing deploy previews of your site**, verify that:

- [ ] your sitemap is being generated correctly
- [ ] your robots.txt is being generated so that it **disallows all requests** in deploy previews

**After going live**, check that:

- [ ] your sitemap is being picked up and read by [Google Search Console](https://search.google.com/search-console)
- [ ] use something like the [Ahrefs Webmaster Tools Site Audit](https://ahrefs.com/signup?plan=awt&return=website-checker) to scan your site and verify that it is indexed correctly and that there aren't any issues.
```

And that's it! You've successfully migrated from the next-sitemap library to Next.js's built-in sitemap and robots.txt APIs.

## See the code changes I made to my site

If you're curious to see the code changes I made to my site to implement this, you can [view the diff on GitHub](https://github.com/mbifulco/blog/pull/894/files).


Migrate from next-sitemap to the Next.js App Directory's sitemap

Transitioning from next-sitemap to Next.js's Built-in Sitemap.xml and Robots.txt APIs

Generating a sitemap.xml file with Next.js

Disallow everything for non-production environments


In this video from my Tiny Improvements series, I'm diving into product analytics to improve newsletter signups on my personal site. Join me as I share my engineering process, from discovering a bug with my old provider to setting up an A/B test with PostHog to find the best-performing newsletter signup form. Despite fewer-than-hoped signups, I've gathered key insights to refine my strategy. Plus, a sneak peek into my next steps, like redesigning the signup popup and enhancing site SEO. If you're a startup founder or indie hacker, you’ll find plenty of actionable tips here!

## See the code changes I made in this video

Code changes for this video are available in this pull request are available at [mbifulco/blog#856](https://github.com/mbifulco/blog/pull/856)

## Timestamps for this video

00:00 Product Analytics for Engineers
01:18 The Results are in!
04:09 Code review in Next.js
04:49 I switched from VS Code to Cursor
05:50 Looking at the variant that won
06:31 Ship a variant directly from PostHog
07:30 A surprise I didn't see coming
08:05 What we did and what's next


The results of my PostHog AB Test are in!

See the code changes I made in this video


In this video, I discuss my recent efforts to improve the product analytics experience on my website. Over the past two weeks, I addressed a sign-up issue by switching to a new email service and implementing AB testing on my newsletter forms. I analyzed the results, but discovered a bug that prevented complete data collection. I walk through identifying the problem, show the fixes I made, and reset the experiment to gather accurate data. I also encourage viewers to check out the open-source code for my site and invite feedback and questions.

## See the code changes I made in this video

Code changes for this video are available in this pull request are available at [mbifulco/blog#840](https://github.com/mbifulco/blog/pull/840)

## Video Timestamps

00:00 Introduction and Recap
00:44 Analyzing AB Testing Results
02:32 Sponsorship Shoutout
03:01 Deep Dive into Data Analytics
05:41 Fixing Implementation Bugs
08:33 Resetting and Restarting the Experiment
10:28 Conclusion and Next Steps


How I found a missing change on my next.js site with PostHog


## Test, trust the data, and optimize your site for better results

In this video, I dive into A/B testing with PostHog to boost newsletter signups on my site. I walk you through my process-from identifying issues with signups, switching email platforms, to experimenting with different UX elements. I'll show you how to set up and run A/B tests, and explain why trusting the data is crucial for making smart, data-driven decisions. If you're a developer or UX enthusiast, this video will help you refine your own website's user experience.

The code for my site is available on github - you can see the Pull Request for the changes associated with this video here: https://github.com/mbifulco/blog/pull/832

## Watch the first two episodes in this series:

1. [Product Analytics for engineers: Debugging a conversion problem on my next.js site](https://mikebifulco.com/posts/debugging-a-conversion-problem-on-my-nextjs-site)
2. [Fixing the bug behind my Conversion Problem](https://mikebifulco.com/posts/fixing-my-conversion-problem)

## Video Timestamps

- 00:00 Introduction and Series Overview
- 00:35 Identifying the Signup Problem
- 00:59 Switching to Resend
- 01:11 Exploring AB Testing
- 01:19 Sponsorship by PostHog
- 02:11 Website Elements and Hypotheses
- 04:07 Implementing AB Tests
- 05:03 Setting Up PostHog Experiments
- 07:20 PostHog Implementation Details
- 08:31 Testing and Analyzing Results
- 10:52 Tell me what you think!

<details>

  <summary>

    <b>Transcript</b>

  </summary>

[00:00:00] **Mike Bifulco:** Hello, and welcome back to tiny improvements. My name is Mike And since we last talked, I've been working on a problem on my website. This is the third video in a series that I'm doing on trying to get people, to sign up for my newsletter.

[00:00:11] If you're new here, I read a weekly newsletter called tiny improvements.

[00:00:13] That is meant for product developers, startup founders, indie hackers, people who are interested in building products. Where I share the things that I've learned, building products that companies like Google and Stripe and Microsoft, and more recently building a venture capital backed startup I'm co-founder of a company called Craftwork. We went through Y Combinator last summer. And so this is my way of giving back. I'm trying to give away the things that I know for free, because I've been really lucky in my career.

[00:00:34] And I want you to be as lucky as I have been. So since you're here, let's talk about where we've been in the first video. I discovered that I had a problem with signups and people weren't subscribing. I was using convert kit for my newsletter at the time and was having some sort of mystery problem with signups.

[00:00:47] So I added a bunch of telemetry to show me when signups were happening and to try and detect failures from that experience, I found that. I was getting some mystery failures with convert kit in my implementation of my newsletter signup form. And that's [00:01:00] what led me to what I did in the second video, which is where I replaced convert kit with Resend.

[00:01:03] So now I use Resend email platform for both sending my newsletters and getting people to sign up for my mailing list. That's been really nice so far. In this video I'm exploring the next thing I wanted to get into, which is AB testing stuff on my site.

[00:01:16] So let's take a look at that and see what that actually means in practice.

## Sponsored by Posthog

[00:01:19] **Mike Bifulco:** Hey, all this video series is sponsored by PostHog. After seeing my first video, they reached out about a sponsorship, which is really cool.

[00:01:25] And frankly, I'm flattered to see it. I was planning on making these videos either way and that's because I really enjoy PostHog. I also really enjoy building products and seeing things improve over time. . I wouldn't take a sponsorship from something that I don't believe in and don't use, I hope you're enjoying this series and I hope you're learning something.

[00:01:39] If you're really into it, it might also be worth giving. PostHog a shot. One of my favorite things about the platform is that they're free tiers, unbelievably generous. You can get a whole lot out of PostHog for free before paying anything to decide if it's worth it for you.

[00:01:50] There's also a self host option. If you want to go that route and PostHog also has a ton of open source software, which means you can have influence over the product itself.

[00:01:58] I think that's really amazing. And the team has been [00:02:00] super responsive so far. So if you want to check it out, go to PostHog dot com slash Mike Bifulco. There'll be a link in the description. It will let them know I sent you and it means a lot to me.

[00:02:09] Thanks so much. I hope you enjoy the rest of the video.

[00:02:11] Okay, this is my site and this is one of the things that I'm interested in testing to see if I can make a difference in the amount of people signing up.

[00:02:17] When you visit my site, read one of my newsletters or any of the articles on my site. There's a bunch of things that are made available at the top of the pages, a cover image and a title things you're probably used to seeing, I have this nice dynamic. Table of contents, which is really helpful I also record audio versions of some posts, which is why we see this podcast embedded here. if we scroll through the post, when you get towards the end of it, a couple of things happen.

[00:02:36] This pop-up, that just flew in from the left is a thing called polite pop. This is where the first call to action typically comes for someone to subscribe to my newsletter. I've got a couple of hypotheses on how we can do better here. What I want to do is test the copy.

[00:02:48] That's rendered here to see if there's maybe a different treatment, that we can show. To people to get them more compelled to sign up. I might also choose to remove this. It kind of is just an ag and I think people are really well-trained to just dismiss pop-ups and [00:03:00] try and get them out of their face.

[00:03:01] And it's a little bit of a rude experience while you're on a site. I see loads of people coming to my site and clicking this. No thanks button. I've got tons of telemetry on that. That may not be long for this world.

[00:03:09] The other thing that I see, and I think this is worth testing straight away is at the bottom of each post.

[00:03:14] I have this giant banner, which is this big striking thing that says ship products that matter. And I really like this. This is something I workshopped for awhile, trying to come up with a slogan or phrase, something that was sort of attention grabbing. That's supposed to give people a reason to sign up.

[00:03:28] However, I think what I've done here is I built something that is so different from everything else. That's that seen on my site that. By the time you get down here, as you scroll down and you get this great big dark banner that appears, it's pretty obvious that the article is over and then I'm not giving you any more value here. And so what I want to test is if I can replace this with something that's a little more subtle.

[00:03:47] So maybe not a giant banner like this, but something that says, Hey, I write this cool newsletter. I think you might like it. Please subscribe. And maybe Tiki into people's psychology a little bit to keep them reading on the page. watching back some replays of people visiting the site. [00:04:00] It's pretty common to see someone get to about here and go away.

[00:04:02] And if they leave before they get down here, I've lost. So. we're going to try a couple of things here and that's. Where AB testing comes in.

## A-B Testing in PostHog

[00:04:08] an AB test is exactly what it sounds like. I have version a version B and sometimes C D E F a number of different versions of one bit of user experience that I want to test against each other to see how they perform. The idea here is to see that if I do this versus that is one going to get me more people signing up for my newsletter or not.

[00:04:24] **Mike Bifulco:** And we can kind of test some hypothesis from there. Now there's some really important statistics involved in how this works. What I really like is that when you collect data, the math will tell a story. And if you're willing to trust the math and listen to what the statistics say, you can probably make a good data driven decision to move yourself from doing well, to doing better and incremental change over time will do really well.

[00:04:46] But you have to be comfortable with trusting what the statistics say. And let me be the first to tell you that there's a lot to learn about that, and it can be really counter-intuitive. I am far from a statistics expert. I have learned a lot of mathematics in my life.

[00:04:57] You have to learn to trust the statistics. [00:05:00] And so that's what we're going to try to do here. Let me get off my high horse. Let's talk about this. So this is my first experiment is called newsletter banner treatments. So you give it a name, which is a human readable thing, and then our feature flag key.

[00:05:10] So this is the key that we'll use programmatically to make decisions based on this experiment. So I gave it newsletter banner treatments as the key. The description here again is a human readable thing. It's meant for me when I go review these later on to see what this experiment was all about.

[00:05:23] So I may fill this in with more detail, but the idea is I want to test different UX presentations of the newsletter banner that appears at the bottom of articles. To see which one performs best.

[00:05:32] By default PostHog will give you a control, which is the fallback value. So the control is meant to be in scientific method speak.

[00:05:39] That means the thing that we know how it performs and the baseline for the experience right now, that's that big dark banner that I show on my site that I was just showing you that people sort of. Seem to scatter away from. I'm running two tests against that in this case. So I'm going to test a simple implementation and a hero implementation. Those are bits of UI that I've built alternatives for.

[00:05:58] And I'll show you what they look like. [00:06:00] So if I come through and I hit, continue on this experiment, we'll see that PostHog does a little bit of thinking. And it goes to do some give me some numbers on roughly what this would look like. And what I want to check is my newsletter subscribed event.

[00:06:11] So I'm hoping to see more of these newsletter subscribed things happening. With one variant of this versus the other, they'll probably all perform slightly differently. We're going to pick the one that wins out of the three. So. I've picked the event that I want to use. If I go to the next page here, what I get is the goal for the experiment.

[00:06:28] So this now shows me everything in one view. What it's saying at the very top is this is a draft. I haven't actually started it yet. By results are not significant, meaning I haven't collected enough data to make a decision. And the feature flags name is here.

[00:06:39] Down here, we get implementation details from PostHog, which is literally code snippets that you can use in your application. So you can switch through these and see what they look like.

[00:06:47] There's react native there's no, there's Ruby, all kinds of stuff in here. What's really cool is they show you exactly how to do this. So you load the feature flag using the feature flags name. If you have a user ID, like an email address, if someone's logged in, you can pass that [00:07:00] into, they always get a consistent experience per user. And then you make a decision based on what comes back. So the feature flag will say for this person render a simple one. It might come back and say for this person render a hero one and you do different things based on that. So what I'll do is I'll hit launch on this thing, It'll sort of thing for a second and go live and I'll, show you what the implementation looks like, next.

[00:07:20] Okay, implementation time. So I'm using post hogs library called PostHog JS.

[00:07:26] Thanks sports, some really nice react components, which I'm using in this implementation here. So this experiment name, newsletter, banner treatments is what we just saw on the site. These are the three different experiment variants.

[00:07:35] So these are the things that the experiment will tell me to render for different users. And we'll collect data on that. And I'm using a thing called PostHog feature, which is a react component exported by post hog, which takes in the name of the flag. As well as the part of the experiment to match.

[00:07:50] So what this is saying is we're going to do this with the newsletter experiment. So if it comes back and says control, we're going to render this thing, the fancy component, which is the newsletter banner that's currently shown on [00:08:00] the site with the dark background.

[00:08:01] I've created two other variants. The simple one here is based on a fairly standard tailwind component library. And this one called hero is one that I was using on other pages on my site, but wasn't consistently showing on the bottom of articles.

[00:08:13] now I'm taking all three of these things. And for about a third of users, I'll show each one of these and we'll track. How many of them convert to newsletter subscriptions over the next. Week or so, and we'll see what comes back.

[00:08:23] Next thing to see is what this looks like in the browser, because PostHog also provides some tools for you to be able to poke around with these and make sure it works well before you roll it out to your users.

## PostHog's A-B Testing tools in the browser

[00:08:31] **Mike Bifulco:** All right back here on post-talk and here's what I want to show you.

[00:08:35] from any page on PostHog, you can get to this handy thing called toolbar. If I click into this, this gives me the ability to launch a PostHog specific toolbar within the context of my site. So it doesn't really fancy injection of content. And it renders PostHog using what I believe is a Chrome extension.

[00:08:50] And so if I click on this, it's going to load my local host, which I've told is where I'm running my site. This is what the tool bar looks like.

[00:08:55] Typically you'll see it. Like this, when it first comes up and we're really interested in this [00:09:00] feature flags. Functionality here. So there's a bunch of things here. You can do actions and heat maps and all kinds of stuff like that. But let's take a look at feature flags first. I'm going to navigate to one of my newsletters, the most recent one, we were just checking this out and I'm going to scroll to the bottom and I'm going to go to this feature flags thing here and already something looks different, This is not the banner that was showing before. And this is a sign that my experiment is working. you can see here, I can manually override the experiments that I'm seeing personally in development.

[00:09:25] And as I toggle through these, I get three different presentations. this one might look familiar. This is the hero one I had before. This is the simple implementation that I wanted to take a look at. And then here's the control implementation. the important thing to test here is we're trying to test things that could produce the same output against each other. When users come to this page, I want to make sure they can submit this form and get the output that I want from each of them, which is that newsletter subscribed event.

[00:09:46] So from this control one, This is a newsletter subscription form. First name and email address is exactly what we want to see the simple form, same thing. First name, email address, and then the hero form. Once again, first name and email address. So. [00:10:00] All the primitives are here. I have three different variants of this thing.

[00:10:03] The next thing to do is to set this live and just record some traffic for awhile. there are some statistical minimums. We have to get a certain amount of users interacting with each one of these, to be able to say whether it worked or not. And the next part is just waiting. So we're going to spin this up on PostHog.

[00:10:17] We're going to collect some data over the next week or two, however long it takes, and then we'll start to make some decisions based on that and go into the next step from there.

[00:10:23] So that's all there is to it. It's a little bit of implementation. It's a little bit of work to set up the three different variants, but now I need to trust the statistics. I need to listen to what comes back from post-hoc as the experiment runs. And tweak my site design based on the data that comes back. My guess is it'll probably take about two weeks for this to come back with meaningful results. But lucky for me, I can run a couple of experiments in parallel, as long as the two different experiments don't interfere with each other. I can probably still draw a valid conclusion from it. So over the next week or so I'll watch this and I'll see what happens.

## What do you think?

[00:10:52] **Mike Bifulco:** If you've been hanging on for the whole series so far, I'm really curious what you think of this.

[00:10:55] Is this something you would find useful for your site? Do you have an idea of experiments you'd like to run? Do you have [00:11:00] questions about this stuff?

[00:11:01] I spent a bunch of my career working in design and user experience. So this sort of thing is a process that I really enjoy, but I can understand that it might be frustrating too, especially if you just want to get to results.

[00:11:10] If you have questions for me, feel free to drop them in the comments. . Your ideas are always welcome here. There's no criticism too big or too small. I'd love to hear what you're thinking. Thanks so much. My name is Mike bi-fold co this is tiny improvements and we'll catch up for the next one.

</details>


Unlocking A/B Testing with PostHog: Improving Newsletter Signups

Test, trust the data, and optimize your site for better results

Watch the first two episodes in this series:

PostHog's A-B Testing tools in the browser


Watch [Product Analytics for engineers: Debugging a conversion problem on my next.js site using @PostHog](https://youtu.be/4nWcB1a8M1s) on YouTube

Thanks so much to PostHog for sponsoring this video and supporting my work! Give it a whirl over at https://posthog.com/mikebifulco and tell 'em I sent you!

## Summary

In part two of my adventure with product analytics, I tackle a significant drop in my newsletter subscribers. Join me as I dive into the data using PostHog, identify and fix critical errors, and ultimately switch from ConvertKit to Resend – a more developer-friendly email platform.

Along the way, I'll show you how I integrated tRPC into my Next.js site for a type-safe API, created hooks for better data management, and set use posthog to chase down the root of the bug.

Whether you're a startup founder or an indie hacker, you'll find valuable insights to improve your product. Plus, transparency is key – this video is sponsored by PostHog, but all opinions are genuinely mine. Let's build something great together!

## Video Timestamps

00:42 ConvertKit Stats

01:06 Subscription Form

02:45 Sponsored by Posthog

03:16 Resend broadcasts

03:36 tRPC

03:54 tRPC Demo

04:43 Subscription form code walkthrough

06:27 useNewsletterStats

07:33 What I'm seeing in posthog

08:37 What's next

09:45 Next PostHog tests

10:11 What do you think I should do next?

10:31 Wrap up

## Links

- Watch part 1 of this series: [Product Analytics for engineers: Debugging a conversion problem on my next.js site using @PostHog](/posts/debugging-a-conversion-problem-on-my-nextjs-site)


Fixing the sign-up problem on my site using PostHog Product analytics


Watch [Product Analytics for engineers: Debugging a conversion problem on my next.js site using @PostHog](https://youtu.be/4nWcB1a8M1s) on YouTube

## Summary

I've been experiencing a problem with my personal website, mikebifulco.com, where my newsletter sign-ups have mysteriously plummeted. In this video, I take you through the process of diagnosing and debugging this issue using Posthog, a powerful product analytics tool.

We'll start by taking a look at the site, identifying potential problem areas, and then diving into the code. I'll show you how I implement Posthog to track user interactions and gather valuable insights. We'll explore how to handle form submissions, track events, and use Posthog’s features to pinpoint what's going wrong.

Throughout the video, I'll share my thought process, hypotheses, and the steps I take to experiment and test different solutions. If you've ever had a non-bug issue in production, this video is for you. Let's get to the bottom of this conversion rate mystery together!

You might be interested in this video if you want to learn about debugging and optimizing website performance, especially using tools like Posthog. Relevant technologies and subject areas include React, Next.js, form handling, product analytics, serverless functions, and conversion rate optimization.

## Video Timestamps

0:00 Product Analytics for engineers: Debugging a form conversion rate problem on my website with Posthog

01:05 Site walkthrough

01:30 Are people annoyed by the popup?

02:53 Long form article pages

04:02 Looking at the code

05:20 Dive into the code

07:13 Newsletter form refactor

10:45 Serverless function for newsletter subscribe

12:26 The results in Posthog

13:29 What we did, and what's next

14:43 Why Posthog and product analytics are helpful

15:41 Epilogue: We learned something! Next steps...

## Links

- Watch [Product Analytics for engineers: Debugging a conversion problem on my next.js site using @PostHog](https://youtu.be/4nWcB1a8M1s) on YouTube
- Part 2 of this series: [Fixing the sign-up problem on my site using PostHog Product analytics](/posts/fixing-my-conversion-problem)


Product Analytics for engineers: Debugging a conversion problem on my next.js site


Watch [The SECRET you need to hear about the difference between Taste and Skill](https://youtu.be/Ym6HQopqe-4) on YouTube

## Summary

Mike discusses how important it is for creatives, especially those building startups and indie projects, to understand this gap and use it as motivation.

If you're on this journey, it is to your benefit to draw from a broad array of influences, and relate it to your creative output.

In addition to the critical step of building your chops by practicing your craft, gathering inspiration from others who excel in the field is a game changer. This is borrowed from my extensive experience in building software. and a career spent learning about the critical role of good design.

If you're aiming to produce better and more user-friendly products, this video offers valuable guidance and inspiration to help you along your creative journey.

## Video Timestamps

0:00 Ira Glass on Beginners
05:35 Mobbin
06:03 Wrap up
06:31 💌 Tiny Improvements, my newsletter

## Links

- Watch [The SECRET you need to hear about the difference between Taste and Skill](https://youtu.be/Ym6HQopqe-4) on YouTube


Your tastes will always outpace your skill, and what to do about it


## Introduction

When working with Supabase, you may need to seed your database with initial data. This is a common task when setting up a new develper environment, or when you need to populate your database with predictable sample data for testing purposes. In this post, I'll show you a simple pattern for seeding your Supabase database using TypeScript.

## Prerequisites

This post assumes you have a [Supabase](https://supabase.com) project set up and a basic understanding of TypeScript. If you're new to Supabase, you can follow the [official documentation](https://supabase.io/docs/guides) to get started.

Although my site is built with [tRPC](https://trpc.io/), [Drizzle ORM](https://orm.drizzle.team/) and [Next.js]https://nextjs.org/, this pattern can be adapted for other frameworks, libraries, and databases.

### Seeding the database with sample data using Faker

This example uses a library called [Faker](https://fakerjs.dev/) to generate fake data. Faker is a popular library for generating random data such as names, addresses, and phone numbers. You can install it using your package manager of choice:

```bash
# note that faker is installed as a dev dependency
pnpm install faker -D
```

## Setting up the seed data

For this example, we will use a simplified Drizzle ORM Database schema with two tables - `chefs` and `recipeTypes`. Here's what the schema might look like:

```ts
// schema.ts

// ...other tables and imports above
export const chefs = createTable('chefs', {
  id: serial('id').primaryKey(),
  name: varchar('name', { length: 256 }).notNull(),
  email: varchar('email', { length: 256 }).notNull(),
  phone: varchar('phone', { length: 256 }),
  address: text('address'),
});

export const recipeTypes = createTable('project_type', {
  id: serial('id').primaryKey(),
  name: varchar('name', { length: 256 }).notNull(),
  description: text('description'),
});

export const recipes = createTable('recipes', {
  id: serial('id').primaryKey(),
  name: varchar('name', { length: 256 }).notNull(),
  description: text('description'),
  recipeTypeId: integer('recipeTypeId').references(recipeTypes.id),
  recipeChefId: integer('recipeChefId').references(chefs.id),
});
```

I try to keep my seed data organized in a `src/db/seed` directory. Each file in this directory contains seed data for a specific table in the database. For example, if you have a `chefs` table and a `recipeTypes` table, you would create two files: `chefs.ts` and `recipeTypes.ts`.

Here's an example of what the directory structure might look like:

```bash
src
└── db
    ├── seed
    │   ├── chefs.ts
    │   ├── recipes.ts
    │   ├── recipeTypes.ts
    │   └── seed.ts   // this is where we collate all seed data and insert into db
    ├── db.ts          // exports the db instance from Drizzle
    └── schema.ts      // database schema de  ition
```

## Creating seed data

This script will stick data into the `chefs`, `recipes`, and `recipeTypes` tables. To keep things neatly organized, each table is represented by its own file. Each of these files exports a function that seeds the data for that table, which is of the type:

```tsx
type SeedFunction = () => Promise<string>;
```

The string returned by each `SeedFunction`'s promise is a message that will be logged to the console when the seeding is complete. I use this to keep track of how many rows were inserted into the database.

### For tables with predefined data

The simplest option is tables with predefined or hand-written data. For these, I more-or-less hand-write objects to stick into the database. Here's an example of how to the `recipeTypes` table:

```tsx
import db from '../db';
import { recipeTypes } from '../schema';
import type { SeedFunction } from '../seed';

type RecipeTypeRow = {
  name: string;
  description: string;
};

// note the export: we will use this later
export const RecipeTypes: SeedFunction = {
  Breakfast: { id: 1, name: 'Breakfast', description: 'Breakfast recipes' },
  Lunch: { id: 2, name: 'Lunch', description: 'Lunch recipes' },
  Dinner: { id: 3, name: 'Dinner', description: 'Dinner recipes' },
} as const;

const seedRecipeTypes = async () => {
  // convert the object to an array of values
  const data = Object.values(RecipeTypes);

  await db.from(recipeTypes).insert(data);

  return `${data.length} Recipe types seeded successfully`;
};
```

Some things to note in the above code:

- We're creating an array of objects with predefined data for the `recipeTypes` table.
- We're inserting the predefined data into the `recipeTypes` table using the `db.from(recipeTypes).insert(data)` method.
- The `return` statement at the end of the function is a string, which will be logged to the console when seeding this table is complete.
- the `RecipeTypes` object is exported so we can use it in other seeding functions.

### Generating data with Faker

For tables that can use randomized data, a slightly different approach is used with the [Faker.js](https://fakerjs.dev/guide/) library, which can generate many different types of randomized data.

Here's how to populate the `Chefs` table with a list of fake chefs:

```tsx
import faker from 'faker';

import db from '../db';
import { chefs } from '../schema';
import type { SeedFunction } from '../seed';

type ChefRow = {
  name: string;
  email: string;
  phone: string;
  address: string;
};

const seedChefs: SeedFunction = async () => {
  // generate random data for 10 people to stick into the chefs table
  const data: ChefRow[] = Array.from({ length: 10 }, () => ({
    name: faker.name.findName(),
    email: faker.internet.email(),
    phone: faker.phone.phoneNumber(),
    address: faker.address.streetAddress(),
  }));

  await db.from(chefs).insert(data);

  return `${data.length} Chefs seeded successfully`;
};
```

### Seeding related data

If you have tables with relationships, you can seed related data by using the `id` of the parent table. For example, if you have a `recipes` table that has a foreign key to the `recipeTypes` table, you can seed the `recipes` table with the `id` of the `recipeTypes` table.

Here's an example of how you might seed the `recipes` table with related data:

```tsx
import db from '../db';
import { recipes } from '../schema';
import type { SeedFunction } from '../seed';
import { RecipeTypes } from './recipeTypes';

type RecipeRow = {
  name: string;
  description: string;
  recipeTypeId: number;
};

const seedRecipes: SeedFunction = async () => {
  const data: RecipeRow[] = [
    {
      name: 'Pancakes',
      description: 'Delicious pancakes',
      // use the id from the RecipeTypes object we exported earlier
      recipeTypeId: RecipeTypes.Breakfast.id,
      // since it can be any chef, we'll just use the first one
      recipeChefId: 1,
    },
    {
      name: 'Spaghetti',
      description: 'Classic spaghetti',
      recipeTypeId: RecipeTypes.Dinner.id,
      recipeChefId: 2,
    },
    {
      name: 'Roast chicken',
      description: 'Juicy roast chicken',
      recipeTypeId: RecipeTypes.Dinner.id,
      recipeChefId: 3,
    },
    {
      name: 'Sandwich',
      description: "Good ol' ham & swiss",
      recipeTypeId: RecipeTypes.Lunch.id,
      recipeChefId: 4,
    },
  ];

  await db.from(recipes).insert(data);

  return `${data.length} Recipes seeded successfully`;
};
```

## Seed the database

Now that we have a script written for each table in the database, we we need to run them each in a specific order, as some tables may depend on others. I keep a file called `seed.ts` which to run each of the seeding functions in the correct order.

```tsx
import seedChefs from "./chefs";
import seedRecipes from "./recipes";
import seedRecipeTypes from "./recipeTypes";

const seedDb = async () => {
  // note: this function assumes we're starting with an empty database

  console.log("Seeding database...");

  console.log("Adding independent data...");
  const res = await Promise.allSettled([
    seedRecipeTypes(),
    seedChefs(),
    // add more independent seeding functions here
    // these will run in parallel
  ]);

  res.forEach((result) => {
    if (result.status === "rejected") {
      console.log("Error seeding database:", result.reason);
    } else {
      console.log(result.value);
    }
  });

  console.log("Adding related data...");
  const dependentTasks = [
    seedRecipes,
    // add more dependent tasks here, they will run in this order
  ];
  for (const task of dependentTasks) {
    try {
    const result = await task();
    console.log(result);
    } catch e {
      console.log("Error seeding database:", e);
    }
  }

  console.log("Seeding complete!");
};

seedDb()
  .then(() => {
    console.log("Seeding complete!");
    process.exit(0);
  })
  .catch((err) => {
    console.error("Error seeding database:", err);
    process.exit(1);
  });
```

You can run this script using your package manager of choice, by adding an entry to `package.json`:

```json
{
  "scripts": {
    // ...etc
    "seed": "tsx --env-file=.env src/db/seed/seed.ts"
    // ...etc
  }
}
```

Then, run the script using:

```bash
pnpm run seed
```

This will seed your database with the sample data you've defined. Note that we're using `Promise.allSettled` to ensure that all the seeding functions run in parallel. This can be useful if you have a large amount of data to seed. It will also catch any errors that occur during seeding and log them to the console.

`Promise.allSettled` returns an array of promises that are settled (either fulfilled or rejected). This allows you to handle each promise individually and log any errors that occur. It is different from `Promise.all`, which will reject the entire promise chain if any of the promises are rejected - you may want to use `Promise.all` if you want to stop seeding if any of the promises fail.

When run, your script should output something like this:

```bash
Seeding Database...
Adding independent data...
3 Recipe types seeded successfully
10 Chefs seeded successfully

Adding related data...
10 Recipes seeded successfully
Seeding complete!
```

## Adapt this pattern to your needs

This is a simplification of the pattern I use in my projects to seed my Supabase database. You can adapt this pattern to suit your needs by adding more tables, more data, or more complex relationships between tables. The key is to keep your seed data organized and to use a consistent pattern for seeding your database.

I use this to reset my database to a known state when I'm working on new features, and to set up my test environment with fresh, predictable data. It's a simple pattern, but it's been incredibly useful for me in my projects.


Seed your Supabase database with this simple pattern

Seeding the database with sample data using Faker

Oh, hello