BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Instagram Notification Management Now Uses Causal Inference Instead of Click-through Rate

Instagram Notification Management Now Uses Causal Inference Instead of Click-through Rate

Bookmarks

Instagram has changed the way of creating notifications, moving from click-through rate (CTR) to causal inference and ML models to identify highly active users who are likely to receive the notifications.

CTR is one the most used algorithms to predict how good content is for a user based on the probability of the click on the notification. If the probability is too low, the notification will be dropped in the sending flow, and the user won’t receive the notification. CTR model-based filtering works well for the daily digest notification, however, a large portion of the notifications are sent to users who are relatively active in terms of Instagram usage. The goal is to provide better notifications to the users and correctly identify these kinds of users, who are relatively active or become active when receiving the notification to reduce the risk of user engagement reduction.

In this way, the problem became a user selection problem: maximizing the notification efficiency by selecting the right users. The solution adopted is a combination of causal inference and machine learning.

Assuming that there is a fixed cost to send the notification and there is a daily budget to send these notifications, it becomes a budget allocation problem. The solution to the problem is to calculate the incremental value to send the notification, rather than not sending it. Mathematically, it can be modeled in this way:

\(ui=Pri(active|do(send notification)) – Pri(active|do(drop notification))\)

For active users, sending the notification could be inefficient and may even spam them. To optimize the budget, the notification can be sorted by the incremental value in descending order, and only the top notifications with high incremental value are sent. In this way the overall incremental values are maximized with the limited sending volume budget.

Estimating the incremental value of a notification before it is sent is a challenging problem. It’s a causal inference problem and uplift modeling techniques can be used to solve it. To apply the uplift model, the Meta engineers developed a randomized experiment in which each notification was randomly sent or not. Based on these data, a neural network is developed based on an uplift model to predict the incremental value of a sent notification.

The online generated score is compared to a given threshold to decide if a notification will be sent or dropped, but from the data gathered a sending rate fluctuation is observed because of the uplift estimated by the ML models. Meta engineers use an online quantile computation service to transform the raw uplift estimates towards a standard uniform distribution while preserving the orders, to stabilize the sending rate.

Instagram decision flow for notification management

Decision flow for notification management 

By applying this model and targeting the users, an improved user experience is reached, decreasing resource usage without a decline in user engagement.

About the Author

Rate this Article

Adoption
Style

BT