InfoQ Homepage Fault Tolerance Content on InfoQ
-
Uber Redesigns M3DB Sharding with Subclusters to Limit Failure Impact
Uber has redesigned shard placement in M3DB with fixed size subclusters to limit the impact of node failures, maintenance, and cluster scaling. The approach bounds shard dependencies, preserves replica isolation, and uses a greedy algorithm to select shard migrations while avoiding a separate rebalancing pass and unnecessary data movement.
-
AWS Cannot Restore Data Held Only in Damaged Middle East Availability Zones
AWS has told customers it cannot restore resources and data hosted exclusively in the mec1-az2 availability zone in the UAE, or exclusively in the Bahrain region, after damage during the conflict with Iran. The company says the Bahrain damage spanned multiple availability zones and exceeded what its regional and multi-AZ services are designed to withstand.
-
Zone Redundancy Comes to API Management Standard v2
Microsoft has enabled zone redundancy on the Standard v2 tier of Azure API Management, following its arrival on Premium v2 in December. Standard v2 starts at $700 per month against $2,801 for Premium v2, but carries a 99.95% SLA rather than 99.99%. Zone redundancy can only be configured when creating an instance.
-
From Minutes to Seconds: Uber Boosts MySQL Cluster Uptime with Consensus Architecture
Uber redesigned its MySQL fleet using a consensus-driven architecture based on MySQL Group Replication, reducing cluster failover time from minutes to seconds. By moving leader election and failure detection into the database layer, Uber improved availability, simplified external orchestration, and strengthened consistency across thousands of production clusters.
-
AWS Simplifies Multi-Region Failover with ARC Region Switch
AWS's Amazon Application Recovery Controller Region Switch revolutionizes multi-region failover with a fully-managed, centralized solution. Simplifying disaster recovery, it automates and coordinates essential tasks across AWS services. With proactive validation and a global dashboard, it transforms complex processes into confident, push-button drills, enhancing reliability and cost efficiency.
-
Amazon VPC Route Server Generally Available, Providing Routing Flexibility and Fault Tolerance
AWS has recently announced the general availability of Amazon VPC Route Server. This new option simplifies dynamic routing in a VPC, allowing developers to advertise routing information via Border Gateway Protocol (BGP) from virtual appliances and dynamically update the VPC route tables associated with subnets and internet gateways.
-
How Google Does Chaos Testing to Improve Spanner's Reliability
To ensure their Spanner database keeps working reliably, Google engineers use chaos testing to inject faults into production-like instances and stress the system's ability to behave in a correct way in the face of unexpected failures.
-
Decathlon Adopts Backend for Frontend (BFF) Pattern to Empower FE Teams
Decathlon established the Backend For Frontend (BFF) architectural pattern as a company-wide recommendation and provided guidelines for its adoption among engineering teams. The four-part series introduces the pattern and explores its benefits and potential pitfalls. The company also shares available alternatives to using the BFF pattern and reviews architectural considerations.
-
Grab Improves Kafka on Kubernetes Fault Tolerance with Strimzi, AWS AddOns and EBS
Grab updated its Kafka on Kubernetes setup to improve fault tolerance and completely eliminate human intervention in case of unexpected Kafka broker terminations. To address the shortcomings of the initial design, the team integrated with AWS Node Termination Handler (NTH), used the Load Balancer Controller for target group mapping, and switched to ELB volumes for storage.
-
Using Code Instrumentation for Fault Injection at the Application Level at eBay
eBay engineers have been using fault injections techniques to improve the reliability of the notification platform and explore its weaknesses. While fault injection is a common industry practice, eBay attempted a novel approach leveraging instrumentation to bring fault injection within the application level.
-
Atlassian Exceeds 99.9999% of Availability Using Sidecars and Highly Fault-Tolerant Design
Atlassian recently published how it exceeded 99.9999% of availability with its Tenant Context Service. Atlassian achieved this high availability by implementing highly-autonomous client sidecars, able to proactively shield themselves from complete AWS region failures. Sidecars query multiple services concurrently to accomplish this goal and ensure that requests are entirely isolated internally.
-
Dealing with Thundering Herd at Braintree
Braintree engineer Anthony Ross explained in a recent article how introducing some random jitter into retry intervals for failed tasks solved a thundering herd issue which was impacting the efficiency of their payment dispute management API.
-
Failsafe 3.2 Released with New Resilience Policies
Failsafe, a lightweight fault tolerance library for Java 8+, launched the major 3.0 release in November 2021. More recently, Failsafe announced the availability of version 3.2 which introduced new Rate Limiter and Bulkhead policies. Failsafe also integrates with asynchronous code like Java’s CompletableFuture.
-
What's New in MicroProfile 5.0
Delivered under the auspices of the MicroProfile Working Group and five months after the release of MicroProfile 4.1, the anticipated release of MicroProfile 5.0 was made available to the Java community. This new release features alignment with Jakarta EE 9.1 and updates to all eight community-developed core APIs and one standalone API.
-
Microsoft Announces Azure Chaos Studio in Public Preview
At the recent Ignite, Microsoft announced the public preview of Azure Chaos Studio, a fully-managed experimentation service to help customers track, measure, and mitigate faults with controlled chaos engineering to improve the resilience of their cloud applications.