Medium Describes "Rex" - a Go-Based Recommendation Service

In a recent blog post, Medium describes how it built a home-grown Go-based recommendation service named "Rex."

As part of its service, Medium displays each user a personalized list of recommended stories (referred to as a "ranked feed"). The original recommendation service was part of the Node.js monolith, and it could only rank 150 stories. However, Medium wanted this service to rank hundreds of thousands of stories per user in under a second. So, they decided to build an entirely new, separate service using Go.

Miles Hinson, a former Medium software engineer, explains the reasoning behind this choice:

As we first made small tuneups to our recommendations system back in March '18, we found that testing and verifying each change was far more time-consuming than we'd like. Bugs would pop up that were difficult to catch in tests and reproduce. It was clear that if we wanted to move more quickly and test new recommendation strategies, we would at least need a major refactor of the code that existed.

According to Hinson, the team decided to give up on Node.js, the prevalent programming platform at Medium. The reason is that Node.js is single-threaded and optimized for simple computational tasks orchestrating asynchronous I/O operations. However, ranking the feed requires some computational heavy lifting that can hog the CPU for quite some time, blocking other requests. This limitation increases the latency for each request, which is suboptimal.

Source: https://medium.engineering/rex-mediums-go-recommendation-microservice-e077bc9582a

They decided to use Go for several reasons. First, Go is multi-threaded and would allow splitting work into separate Goroutines and reduce latency. Second, Go is opinionated and believed to be easy to learn by newbies. Also, Medium has had some previous experience with Go, so some knowledge of the language already existed in-house.

Medium's recommendation feed is composed of several basic steps illustrated in the diagram below. Each step is extensible and allows plugging in additional rules as new requirements come in.

Source: https://medium.engineering/rex-mediums-go-recommendation-microservice-e077bc9582a

1. Aggregating - gather relevant stories for a user from various providers: publications and authors the user follows, topics the user follows, recommendations based on the user's reading history, etc. The collection of providers gets expanded over time, and Medium needed to modify them in the future easily.
2. Preprocessing - filter out stories that may not be suitable for a user at a given time. E.g., a story that a user has already read. Much like aggregation providers, preprocessing rules are volatile and change over time.
3. Annotating - enrich possible stories with additional data needed to rank them from various Medium data stores. Most features are calculated via offline Scala jobs and loaded during feed creation. However, some data still needs to be checked online.
4. Ranking - transform the results from the annotation step into an array of numerical values and pass each story and set of values to another Medium microservice that hosts our feed-ranking models. This separate microservice assigns a score to each story.
5. Postprocessing - ensure a better user experience over the raw output from the ranking models. For example, a business rule that diversifies the ranked list sources ("prevent a single entity from dominating the top of a user's feed").
6. Caching - store the ranked feed in Redis for a certain period.
7. Validating - apply additional business rules on cached feeds, and invalidate "stale" entries in the feed. E.g., remove stories whose authors the user unfollowed in the meanwhile.