Mobius: Adopting JSX While Prioritizing User Experience

Posted by on November 4, 2021

Keeping up with the latest developments in frontend architecture can be challenging, especially with a codebase the size and age of Etsy’s. It’s difficult to balance the benefits of adopting newer tools with the cost of migrating to them. When a total rewrite isn’t possible, how can you ensure that old patterns can coexist with new ones, without totally sacrificing user and developer experience, or loading so much additional code that it harms frontend performance?

This frontend architecture dilemma is one we’ve really wrestled with on Etsy’s Frontend Systems team. Our team is responsible for setting frontend best practices for our product engineers and building infrastructure to support them, and our primary focus for the past few years has been developing a new approach to building Javascript-driven user interfaces on Etsy.

We’ve developed a frontend architecture that we think is an effective, pragmatic compromise. We render static HTML using PHP by default, while giving developers the ability to opt into using “component islands”: self-contained, server-side rendered JSX components used only where complex interactivity is needed.

We named this new architecture Mobius, and the rest of this post will be an overview of what we built (and why!), and how it works.

Javascript at Etsy, pre-Mobius

This work began in 2018, when we saw increasing evidence that the frontend development patterns recommended for buyer-facing website features were no longer meeting product engineers’ needs. This concern was specific to buyer features because, historically, Etsy has used very different frontend approaches across the site.

Our marketplace is two-sided—we build a product for both buyers and sellers—and our code reflects this divide, even though all of it is built in the same large monorepo. Pages for buyers are generally static and informational, but sellers managing their shops and fulfilling orders need something more like a desktop application, to provide a low-friction experience while handling a lot of often-changing information. React is a perfect fit for building more complex applications like this, so we had already begun using it for our seller tools years before, in 2015. The performance and code complexity tradeoffs of a React single-page app were a good fit there. 

On the buyer side, where initial page load performance, mobile-friendliness and SEO are higher priorities, a single page app was less appropriate. We explicitly asked product engineers to avoid using React in that context, which meant buyer-facing pages had a much more limited set of frontend tools. Older code was written in jQuery, with newer features being built with modern, ES6+ transpiled Javascript. Most of the markup for buyer features was generated on the server with our internally-built PHP templating system, with small amounts of Javascript for adding limited interactivity.

This is still a totally viable approach to web application development: good SEO and web performance, as well as progressive enhancement, are baked in. However, it was starting to cause friction for Etsy product engineers, for a few reasons:

Given all this, being able to use JSX components everywhere in our codebase was one of our most common frontend infrastructure requests from product engineers. It no longer seemed reasonable to stick with our jQuery status quo.

Balancing user experience and developer experience

Supporting JSX components in our buyer frontend made sense, but it wasn’t feasible to rewrite every view in our codebase (we have over two million lines of Javascript code!). Anything we built would have to permanently coexist with our PHP-rendered views. Also, we weren’t happy with the tradeoffs of the “easy” solution of adding client-rendered components:

We dealt with the library-size issue by choosing to render our components with Preact, a React alternative that implements the same API in a much smaller codebase (5kb gzipped, as opposed to ~30kb for React).  We were hesitant about this decision at first because of concerns over long-term API compatibility with React. It felt like a risk to rely on the developers of Preact keeping up with future changes to the React ecosystem, and we worried about being locked into using a less standard library if the implementations drifted apart and our needs changed.

Ultimately we decided the benefits of Preact were worth it. As it happened, even in our large codebase the React compatibility issues turned out to be quite limited in scope. The Preact core team was also extremely responsive to our needs, and their legacy-compat package allowed us to support newer React APIs, like hooks, without rewriting all of our existing code.

Preact ended up working out so well for us that we have migrated our entire codebase to it, including our five-year-old seller tools single-page app. In addition to the performance benefits, Preact’s better backwards compatibility made it easier for us to migrate our React 15 code to Preact v10 than to React 16.

Rendering JSX on the server

Given that most of Etsy’s backend view rendering code is written in PHP, a major architectural problem for adopting server-side rendering was determining how to execute our JS views on the server and get that markup into our PHP view rendering pipeline.

Fortunately for us, we already had a lot of the pieces in place to make this possible. The internal API framework we’ve used for many years relies on curl to make multiple API requests in parallel, and then passes that aggregated data on to our view layer. It doesn’t matter to curl how that HTTP response is generated, so it was possible to also make requests to a Node service to fetch our rendered JSX components as well as our API data without major architectural changes.

Although the idea is straightforward, the implementation was not that easy. Our API request code could handle asynchronous processes, but our view rendering layer assumed synchronous execution, so first it had to be refactored to handle waiting for HTTP rendering requests without blocking. (This was a significant effort, but since it’s very specific to our codebase and this blog post is focused on the frontend architecture, we’ll skip the details here.)

Once we enabled asynchronous rendering on the server, we built a Node service to handle the actual component rendering. Our initial proof of concept used Airbnb’s Hypernova server, and though we eventually replaced it with a homegrown service, we kept the general design approach.

Our PHP views are structured as a tree of individual view classes (not unlike a JSX component tree). Typically the templates for these views use Mustache; our work made it possible for leaf nodes of the view tree to provide a JSX component as a template instead. A PHP view that does so looks something like this:

<?php
class My_View {
  const TEMPLATE = 'path/to/my/component.jsx';

  public function getTemplateData() {
      // this data is passed as props for the initial component render
      return [
          'greeting' => 'Hello Code as Craft!',
          'another-prop' => true,
      ];
  }
}

When the PHP view renderer finds one of these JSX views in a tree, it sends a request to our Node service with the name, asset version and template data of that component, and the service returns the rendered HTML. That HTML is combined with all of the PHP rendered HTML and recursively concatenated to build the final document. 

To provide the right component code to the Node service, we keep a manifest of all server-rendered components and build them into a bundle of those components from our monorepo at deploy time. Each version of this bundle is accessible on our asset hosts, and the Node service fetches the bundles as needed to respond to requests, as well as caching the component class itself in memory to save time on future requests.

Component islands in the client

We needed an architectural pattern in the client that matched our “render mostly static markup in PHP, opt into JSX when you need it” approach. Jason Miller and Martin Hagemeister from the Preact core team met with us to discuss their previous solutions to this problem. During that meeting, Katie Sylor-Miller, our frontend architect, dubbed this pattern “component islands”: floating pieces of interactive, JSX-powered UI within an otherwise static page.

Shared code finds each island on the page and hydrates the component, typically when it is about to be scrolled into the viewport, which saves some time executing JS on the initial page load. Hydrating lets us avoid a redundant render on the client the first time we execute our JSX code there. The client code instead finds the server-generated markup (based on a unique identifier) and initial data (inlined into the document as JSON) and “adopts” it, meaning there’s no additional DOM mutation until an update is triggered by user interaction.

Initially, we assumed islands would be generally isolated from each other and not rely on shared state with other components. As engineers began to build features with Mobius, we realized the reality was more complex, especially when features were being incrementally migrated to JSX. There was a clear need for a standard way for islands to communicate with each other and with preexisting Javascript code we did not want to refactor.

A block wireframe of the Etsy listing page. The content is divided into two main columns. The left column contains blocks that represent the listing image carousel. They are outlined in a purple dashed box labeled "Non-island plain Javascript". In the right column, some of the blocks, representing text and a select input, are outlined in orange and labeled "Preact component island".

We considered writing our own event bus to handle cross-island communication, but ultimately decided to use Redux instead. Initially we were hesitant to adopt Redux, because it seemed like too much overhead for the limited complexity of most component islands. After working with product engineers on some early features using Mobius, however, we realized the benefits of familiarity and consistency with our single-page app (which uses Redux extensively) made it the right choice.

We created a global Redux store to serve as a way to share data across these components. JSX components connect to the store using Redux Toolkit. Plain Javascript code can also subscribe to store changes and treat it as more of an event bus, while still having access to the same data, allowing us to incrementally rewrite features while still integrating with existing jQuery-based components. Reducers can be lazy loaded as well, to continue to keep initial JS execution time low.

Component islands can connect to Redux using standard Redux Toolkit utilities like createSlice and patterns like using Connect components and selectors to provide access to the store.

const counterSlice = createSlice({
  name: "counter-slice",
  initialState: {
    count: 0,
  },
  reducers: {
    increment(state) {
      state.count++;
    }
  },
});

To allow existing code to access the Redux store from outside of a component context, we created helper methods for subscribing to store changes.

ReduxStoreManager.subscribeTo(
  "counter-slice",
  (sliceState, prevSliceState) => {
    const { count } = sliceState;
    console.log(`The count is now ${count}.`);  }
);

// dispatch an action to update the state
ReduxStoreManager.dispatch({ type: "incrementCount" });

Building the best tool for the job

Mobius has been successfully powering features in production on Etsy.com since late 2020. We’re proud of how it has enabled us to improve developer experience and velocity for product engineers without rewriting huge portions of our frontend codebase or having a large negative impact on frontend performance.

Now that we have this fundamental tooling in place, we’ve begun to explore additional ways to use Preact and server-rendered Javascript effectively. Being able to use JSX everywhere will make it possible to soften the boundaries between our seller- and buyer-facing features. We hope to use SSR to improve performance in our shop management application, and to use Preact as the primary renderer for our design system components (since currently we maintain two implementations, JSX and plain Javascript).

While building out this entire infrastructure was a major undertaking for us, given our large codebase and preexisting systems, the general principle of only using complex Javascript in the client where you need it is widely applicable, and one we hope can be more widely adopted without quite as much effort.

JSX is often seen as an all-or-nothing choice, but we built a hybrid architecture that lets us have the best of both worlds. We integrated server-rendered JSX with our existing PHP view framework so product engineers can use whichever tool is best for their use case without sacrificing performance.

This work was truly a collaborative effort between many Etsy engineering teams. I particularly want to acknowledge the contributions of the entire Frontend Systems team past and present (Steven Washington, Miranda Wang, Sarah Wulf, Amit Snyderman, Sean Gaffney), as well as Katie Sylor-Miller. Also thank you again to the Preact core team for the initial guidance towards this architecture and all the continued support!

Posted by on November 4, 2021
Category: Uncategorized Tags: , , ,

Related Posts

2 Comments

The Island architecture is relatively new on the market and thus very interesting. Thank you for the detailed blog post!

Do you have any plans on open-sourcing e.g. the lazy-init code (“Shared code finds each island on the page and hydrates the component, typically when it is about to be scrolled into the viewport, which saves some time executing JS on the initial page load”)?

    We considered open sourcing the island hydration code, but there’s not really a lot to it, especially once we remove the things specific to our codebase. It relies on IntersectionObserver to call Preact’s hydrate method, so you could likely write something similar yourself relatively easily.