BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Practical API Design Using gRPC at Netflix

Practical API Design Using gRPC at Netflix

This item in japanese

Bookmarks

A two-part series of articles about API design at Netflix was recently published in the company's technology blog. It describes how they solved the problem of ignoring irrelevant message fields in their backend API requests and responses by using the special Protocol Buffers (Protobuf) message FieldMask.

Alex Borysov and Ricky Gardiner, senior software engineers at Netflix, note that API clients often do not use all the fields present in the responses to their requests. This transmission and computation of irrelevant information for one specific request can waste bandwidth and computational resources, increase the error rate, and increase the overall latency.

The authors argue that such waste can be avoided when API clients specify which fields are relevant to them with every request. They point out that this feature is present out of the box with API standards such as GraphQL and JSON:API and question whether Netflix's wide usage of gRPC in the backend could benefit from an identical mechanism.

They found that a particular message called FieldMask is defined in Protobuf, the underlying message encoding of gRPC. When included in API requests, it allows clients to list which fields are relevant and can be applied to both read and modify operations.

Selecting a few fields from a large message body.
Source: https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518

The authors also describe the pattern chosen for managing the evolution of each API's data model while maintaining some degree of backward compatibility. Old fields are deprecated in favour of new ones, and they advocate for requiring a valid field mask or a version number with every update operation.

The need to optimise the overall footprint of API traffic is also felt in different domains. Others have also concluded that it is beneficial to allow clients to control which data is transmitted. As the consultant Adam DuVander points out in his earlier article about API request flexibilisation, "a bit too much or too little data may seem trivial, but in aggregate they can have a huge impact". DuVander shows how different API implementations addressed this issue.

Detailed code and Protobuf message examples are provided in the article to illustrate how FieldMask messages are used at Netflix.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT