Cloud Follow 330 Followers

Amazon S3 Increases Request Rate Performance and Drops Randomized Prefix Requirement

by Steffen Opel Follow 4 Followers on  Oct 11, 2018

Amazon Web Services (AWS) recently announced significantly increased S3 request rate performance and the ability to parallelize requests to scale to the desired throughput. Notably this performance increase also "removes any previous guidance to randomize object prefixes" and enables the use of "logical or sequential naming patterns in S3 object naming without any performance implications".

DevOps Follow 971 Followers

Facebook Open Sources LogDevice - a Distributed Data Store for Log Storage

by Hrishikesh Barua Follow 15 Followers on  Sep 30, 2018

Facebook open sourced their internal distributed log storage project called LogDevice. It offers high write availability using replication, durable log storage and recovery from failure.

DevOps Follow 971 Followers

How Coinbase Handled Scaling Challenges on Their Cryptocurrency Trading Platform

by Hrishikesh Barua Follow 15 Followers on  Aug 12, 2018

Coinbase, a digital currency exchange, faced scaling challenges on their platform during the 2017 cryptocurrency boom. The engineering team focused on upgrading and optimizing MongoDB, traffic segregation for hotspots to resolve them, and building capture and replay tools to prepare for future surges.

Development Follow 684 Followers

Hyperledger Adds "Caliper" to Measure Blockchain Performance across Implementations

by Kent Weare Follow 11 Followers on  Apr 06, 2018

On March 19th, Hyperledger announced Caliper has been accepted by the Technical Steering Committee as a Hyperledger project. Hyperledger Caliper is a blockchain benchmark tool that allows projects to consistently track performance characteristics across different blockchain implementations.

DevOps Follow 971 Followers

How Uses Kubernetes for Machine Learning

by Manuel Pais Follow 9 Followers on  Apr 01, 2018

Sahil Dua explained how was able to scale machine learning (ML) models for recommending destinations and accommodation to their customers using Kubernetes, at the QCon London conference. In particular, he stressed how Kubernetes elasticity and resource starvation avoidance on containers helps them run computationally (and data) intensive, hard to parallelize, machine learning models.

DevOps Follow 971 Followers

Handling Traffic Spikes from Global Events at Facebook Live

by Hrishikesh Barua Follow 15 Followers on  Feb 26, 2018

Facebook Live’s engineers talked about how they scale their systems to handle traffic from both predicted and unpredicted events. While the latter is handled by their global distributed architecture, the former involves careful advance planning and load testing.

AI, ML & Data Engineering Follow 991 Followers

Smart Replies for Member Messages at LinkedIn

by Andrew Morgan Follow 3 Followers on  Oct 30, 2017

LinkedIn has launched a new natural language processing (NLP) recommendation engine which is used to provide members with smart-reply recommendations to messages. The models and infrastructure development process has been documented in detail in a recent blog post by the engineering team.

Architecture & Design Follow 2408 Followers

GoTo Copenhagen 2017: How Shopify Powers Online Commerce

by Andrew Morgan Follow 3 Followers on  Oct 30, 2017

Simon Hørup, senior product engineering lead at Shopify, gave an overview of how Shopify is architectured to support large sales at GoTo Copenhagen 2017. This included their OpenResty configured NGINX instances, shop and pod isolation architecture, failover strategies and more.

DevOps Follow 971 Followers

AWS Offers 4TB Memory Virtual Machines

by Aymen El Amri Follow 1 Followers on  Oct 10, 2017

AWS has now the largest cloud virtual machine in terms of memory, after launching the x1e.32xlarge, a new memory-optimized EC2 instance type. AWS customers can use this new type of instances in their production environments in order to handle the high memory requirements of software like SAP HANA and their in-memory databases.

Architecture & Design Follow 2408 Followers

Personalized Notifications at Twitter

by Andrew Morgan Follow 3 Followers on  Jun 30, 2017

Gary Lam, staff engineer at Twitter, spoke about personalized notifications at QCon London 2017. This involved giving a high-level overview of their personalization and recommendations algorithms, and an explanation of how they work at scale despite the large volumes of data and bi-modal nature of Twitter.

Architecture & Design Follow 2408 Followers

Real-Time Notifications at Twitter

by Andrew Morgan Follow 3 Followers on  Jun 20, 2017

Saurabh Pathak, engineering manager at Twitter, discussed the site's notification architecture at QCon London 2017. This included highlighting key challenges which are unique to Twitter, such as the bimodal nature of the social network, dealing with spikes, and the requirement to serve notifications in real time.

Development Follow 684 Followers

Microsoft Has Moved the Windows Code Repo to Git

by Abel Avram Follow 11 Followers on  May 25, 2017

Microsoft has completed the move of their Windows codebase from Source Depot to Git using GVFS.

Culture & Methods Follow 798 Followers

Building a Scalable Minimum Viable Product

by Ben Linders Follow 28 Followers on  Sep 29, 2016 1

Scalability should be considered when developing a Minimum Viable Product (MVP). An MVP needs to be technically scalable and you need to have a plan on how to scale quickly when your MVP attracts many users and becomes successful. Knowing your possible performance bottlenecks and using common sense while developing your MVP will get you very far, says Erik Duindam, CTO at Unboxd.

Architecture & Design Follow 2408 Followers

How GitHub Designed its New Load Balancer

by Sergio De Simone Follow 18 Followers on  Sep 26, 2016

GitHub has been at work for the last year to develop a new load balancer, the GitHub Load Balancer (GLB), aimed to be able to scale to billions of connections per day running on commodity hardware. GitHub engineers Joe Williams and Theo Julienne explain how GLB was designed.

Architecture & Design Follow 2408 Followers

How and Why Etsy Moved to an API-First Architecture

by Thomas Betts Follow 48 Followers on  Aug 02, 2016

At QCon New York 2016, Etsy software engineer Stefanie Schirmer told how her company successfully transitioned to an API-first architecture that supports multiple devices, addresses server-side performance problems, and was quickly adopted by development teams.

Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you