BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News BBC Online Uses Serverless to Scale Extremely Fast

BBC Online Uses Serverless to Scale Extremely Fast

This item in japanese

Bookmarks

In a series of blog posts published recently, Johnathan Ishmael, lead technical architect at BBC, explains why BBC Online uses serverless and how they optimize for it.

According to the author, BBC Online uses AWS Lambda for most of its core implementation due to its ability to scale extremely fast. The BBC website can reach 60 million browsers a day, with users requesting up to 20k pages per second. When a breaking news story erupts, traffic can increase 3x in a single minute and then keep rising after that. As Ishmael explains:

These key moments are critical for the BBC, and they are the moments in which the audience turns to us. We must not fail. Any technology we choose must be able to respond to these traffic patterns.

BBC engineers use serverless for two high CPU intensive areas of the application that require unpredictable scaling. The first is the React app that server-side renders the HTML, and "the second is the business logic layer, which transforms data from many different BBC systems into a common data model." They are running over 100 million serverless function invocations per day.


Source: https://medium.com/bbc-design-engineering/optimising-serverless-for-bbc-online-118fe2c04beb

As shown in the simplified stack above, caching is done between the Lambda layers. It is performed to reduce function chaining, where one Lambda calls another Lambda and then waits for it to complete. This chaining incurs additional costs as Lambdas are charged proportionally to their execution time, even when they are idle. Another reason is to allow the backend serverless function to refresh the cache in the background without a user request having to wait for it.

Ishmael states that serverless functions are ideal for handling unpredictable traffic volumes since cloud providers operate these at a large scale. Virtual machines (VMs), on the other hand, are useful for workloads that change slowly or tolerate delay in scaling. Their experience taught them that they need to overprovision by 50% to deliver enough capacity to manage scale events using VMs. Containers can scale up or down more quickly than VMs; however, they still rely on the underlying compute hardware capacity, which would still scale slowly in the face of rapidly rising demand.

An added benefit of using serverless is the reduction of operational complexity of running the service. Much of this burden is offloaded to the cloud vendor since the servers are abstracted away. This reduction in complexity allows the development team to focus more on the product and its experience.

To optimize the lambda functions' performance and reduce costs, BBC engineers performed several optimizations. One optimization is selecting the correct memory profile. According to their tests, "there is a critical point at which further increases in memory have no improvement in performance, but do increase in cost." Another optimization is the utilization of cold start times:

One advantage of the cold start process is that you can take care of any initialization required for when the function executes. This allows the actual invocation to be as efficient as possible. You also don't pay for the cold start period, so essentially a few free cycles you can use to optimize the performance of your function invocation later. We used this time to establish network connections to all the APIs needed (removing the need for SSL negotiation later). We also loaded any JavaScript requirements into memory for use later. This means that we do not have to perform this logic during every execution.

According to the author, one of the disadvantages of serverless is the lack of tools to diagnose issues remotely. As shown in the screenshot below, the BBC engineers use a combination of X-Ray and CloudWatch to diagnose and troubleshoot their services.


Source: https://medium.com/bbc-design-engineering/optimising-serverless-for-bbc-online-118fe2c04beb

This explanation is in continuation to BBC's announcement two months ago, in which they announced that they based their application on serverless computing.

Rate this Article

Adoption
Style

BT