Meta's Engineering team has published a post explaining how they built the infrastructure for Threads, their new online social media and networking service.
On July 5, 2023, Meta unveiled Threads, a new addition to its app family, after a short development period of just five months using a small team of engineers. The decision to launch was abrupt, with the infrastructure teams having just two days' notice. However, the teams were confident in Meta's infrastructure's maturity and past performance to effectively support the app's rapid growth.
The success of Threads' launch was attributed to over a decade of infrastructure and product development, not specifically tailored for Threads, but designed over Meta's lifetime for various other products such as Facebook and Instagram. The infrastructure - built for scale, growth, performance, and reliability - exceeded expectations as Threads was an unprecedented success with over 100 million sign-ups within the first five days.
The infrastructure supporting Threads involves many components, but two key elements are highlighted in the article: ZippyDB, a distributed key/value datastore, and Async, an asynchronous serverless function platform.
ZippyDB played a vital role in scaling keyspaces for Threads. This distributed key/value database - fully managed by Meta's infrastructure team for engineers - allows flexible scaling and placement across multiple data centers. Running on Meta's overall capacity management framework allows the team to add thousands of machines to the cluster within a few hours, though Meta's implementation of ZippyDB has features enabling seamless scaling on short notice too. ZippyDB's multi-tenancy and strong isolation features enable this, together with semi-automated levers to exploit underused existing hosts.
The ZippyDB team exploited the resharding protocol built into ZippyDB to provide horizontal scaling with zero downtime, and without losing consistency nor correctness, even when facing a 100x load spike. This was crucial in preventing interruptions and in ensuring efficient capacity usage in the days following the launch as the platform grew virally.
Async, also known as XFaaS, played a critical role in scaling workload execution for Threads. This serverless function platform processes trillions of function calls daily, and supports multiple programming languages. Async abstracts away infrastructure deployment details, allowing developers to focus on core business logic while exploiting the platform's hyperscale attributes. It efficiently handles computationally expensive operations, such as user onboarding and social graph execution, during Threads' rapid user growth.
Async's ability to defer activities that don't need to block the user's experience to off-peak hours played a crucial role in Threads' success, by making sure that intensive but non-urgent actions were performed asynchronously and at non-peak times. It allowed users to build communities quickly, offering scalable execution for computationally intensive tasks involved in social graph operations. This was especially useful for two scenarios which happened frequently post-launch. Firstly, when celebrities joined Threads, gaining millions of new followers very quickly, and secondly when Instagram users chose to follow the same set of users they follow on Instagram. Both operations require extensive processing, and Async is able to do that by adjusting the execution flow according to capacity.
The combination of ZippyDB and Async were powerful tools in Meta's rapid launch of Threads, combined with the pre-existing robust infrastructure and engineering culture. The shared infrastructure which has been developed over time empowers product teams to move quickly and scale successfully with load redistribution, load balancing and transparent scaling all built-in. The engineering culture emphasizes speed, collaboration and ownership, all of which were critical in coordinating the successful launch of Threads within a short timeframe.