What it's like to migrate a high-traffic website from Gatsby to Next.js

APIs You Won't Hate is a resource for developers who want to learn how to build APIs that are secure, performant, and easy to use. I've been helping to run and maintain the site since roughly the beginning of 2018, when I rebuilt it from a Jekyll site to a static site with Gatsby. This was a huge leap forward for the site - as we were growing the community and adding resources for API developers regularly, it became apparent that we were going to need to support some features that I wasn't really interested in building on top of Jekyll - support for multiple authors, an image optimzation solution.

So, I built the site with Gatsby, as a visual clone of the Jekyll site. It worked fantastically for us, but as we added more and more articles and authors to the site, it became challenging to maintain, for a variety of reasons.

1. Building the site was slow

   From a clean cache, it could take from 11-15 minutes to build, which meant that regular deployments to our hosting provider were starting to get expensive. It also meant that any debug tasks which required a production build were intolerably slow, and very frustrating.

2. Dealing with images was challenging

   Gatsby has an amazingly clever strategy for image optimization, which involves sending all images used on the site through a pre-build pipeline that makes them accessible via Gatsby's graphql API. This is a foundational part of how Gatsby sites are meant to run, and it gives plenty of options for image loading strategies and optimization approaches.

   What we never quite got right with our implementation was a strategy that would allow authors to upload images to the apisyouwonthate.com repo with a naming strategy that was sensible.

   Essentially, due to a quirk in our chosen implementation, each image on the site had to have a unique name, and if two images with the same name were uploaded, it caused all kinds of problems. You can imagine that in the course of writing dozens of articles about APIs, there's more than one occasion to create an image called devtools-network-tab.png, or cover.png, etc.

3. Updating the site required too much understanding of Gatsby's underlying platform

   This basically meant that of the three of us who maintain the site, I was the only one equipped to make significant changes, because I had spent a good deal of time in my day-to-day job building prototypes with Gatsby for Gymnasium. When either of my teammates wanted to make changes to the site, they had a choice: spend hours and hours trying to figure out how the site was configured, struggle to make changes, and submit a PR, or create an issue on GitHub and assign it to me to fix. You can probably guess which happened more often - and looking back, it makes total sense.

4. Some lugnut* had used boostrap to design the layout for the site.

   (*It was me.)

   I used bootstrap out of convenience, and hamfisted it into working with our React site using a couple of libraries and band-aids to get it all "working" properly. This resulted in a near-infinite supply of teeny tiny little layout bugs. These would be fantastic if I needed them for job security... but in practice they were relentless headaches.

Let me be the first to tell you that it's highly likely that everything I accomplished in the site rebuild with Next.js is possible with Gatsby (or your favorite framework). This rebuild sets the stage for a number of features to come, and should make it easier for us to collaborate with new authors, iterate on site designs, and add small features without having to suffer a 10+ minute CI process.

This rebuild gave me a chance to add features, and add some meaningful updates which had been on my plate for quite a long time. Here's a quick summary of some of my favorite features of the new site:

For the first time ever, Open Graph Images are embedded correctly across the site - the site uses a fairly simple strategy, grabbing a predetermined cover image of just-the-right-size and embedding that in the header of each page using Open Graph metadata.

We've also got fairly fine-grained control of other SEO metadata, including author names, article titles, and descriptions. Our new implementation will allow us to set Canonical URLs for articles crossposted to APIs You Won't Hate from other sources as well, which makes for better SEO all around. This may also help down the line if we want to add sponsored posts to the site which point to another origin as the canonical home for the information we share.

Want to see how it works? Dive into the SEO component in the site's source code.

We had previously been using Netlify Forms for the site's contact form, but since the new site is hosted on Vercel, that is no longer possible. We're now using Airtable's API for the form, which lets us collate all submissions into one place. New entries to this form trigger a routine which notifies site owners that we got a message. We then use Airtable to track responses, and make sure that nothing gets lost in the mail.

This is all made possible by Next.js API routes. These are serverless functions that are magically deployed to the cloud for any route at /pages/api/[slug]. We use a similar approach for our Newsletter signup form, which adds contacts to a list on mailjet. You can check out the API code here, and the newsletter form code here.

Our Lighthouse scores shot up with this reimplementation - and while they're a moving target for a variety of reasons, performance on the site is generally really great. This is helped immensely by Next.js's Image component, which uses a variety of web standards and recommended practices to load the right-size image depending on readers' given device and browser.

One of the most important litmus tests available for our site's general performance comes from Google's Search Console. Thankfully, since the site was deployed, we've seen consistent traffic on across all of our pages, with a slight-but-steady increase week to week:

Next also made it far easier to generate a sitemap.xml for APIs You Won't Hate. In Gatsbyland, we needed to use gatsby-plugin-sitemap, which is configurable, but is generated at build time. This meant that debugging sitemap changes took a full 10min+ for each change - which was completely untenable for us.

By contrast, our sitemap implementation with Next.js uses a library called next-sitemap. It is also generated at build time, but in a local dev environment builds take only a few seconds, allowing me to nail our sitemap much more quickly.

Because this implementation work gave me an opportunity to rip out the old site's visual design, I was able to make some sorely changes to the site's overall look and feel. I removed Bootstrap and react-bootstrap, and used Chakra UI for layout building instead. There's a bit of a learning curve with Chakra (like any other UI library), but now that I'm familiar with it, I can build new responsive site pages efficiently and reliably.

Here's some side-by-side comparisons of the old and new site:

Home Page

Book feature page

Blog landing page

We've been up and running on Next for a few months now, and everything has been working smoothly. I'm really happy with how this implementation work went, and am really lucky to be able to have a wealth of choices in amazing developer communities that surround JAMstack and SSG tools. Having now built sites and companies with create-react-app, Gatsby, next.js, and Remix Run (keep an eye out for this one!) -- I can say with a high degree of confidence that they are all super tools. You can get your work done in any one of these libraries - often the challenge we face as developers is sticking with our bets long enough to see them through.

Keeping APIs You Won't Hate running and growing has been a forcing function to build a site that is performant and dependable, and affects a large-scale audience... for me, that's more than enough to motivate the work it takes to keep the site running. The community we've built and the knowledge we're sharing are truly fantastic, and it's my privilege to build real sites that people (hopefully) don't hate.