InfoQ Homepage Infrastructure Content on InfoQ
-
Uber Open-Sourced Its Highly Scalable and Reliable Shuffle as a Service for Apache Spark
Uber engineering has recently open-sourced its highly scalable and reliable shuffle as a service for Apache Spark. Spark is one of the most important tools and platforms in data engineering and analytics. It is shuffling data on local machines by default and causes challenges while the scale is getting very large. Shuffle as a service is a solution developed at Uber for this problem.
-
Google Introduces Zero-ETL Approach to Analytics on Bigtable Data Using BigQuery
Recently, Google announced the general availability of Bigtable federated queries, with BigQuery allowing customers to query data residing in Bigtable via BigQuery faster. Moreover, the querying is without moving or copying the data in all Google Cloud regions with increased federated query concurrency limits, closing the longstanding gap between operational data and analytics.
-
Developing and Evolving SaaS Infrastructures for Enterprises
SaaS companies that are focused on the enterprise market need to evolve their infrastructure to meet the security, reliability, and other IT requirements of their customers. IT admins and large customers are two important sources of requirements to drive development.
-
Google Expands the Tau VM Family with Arm-Based Processors
Recently, Google announced its Cloud Virtual Machines (VMs) based on the Arm architecture of Compute Engine called Tau T2A. These VMs are the latest addition to the Tau VM family that offers VMs optimized for cost-effective performance for scale-out workloads and are available in preview.
-
AWS Announces AMD Based R6a Instances for Memory-Intensive Workloads
AWS recently announced the general availability of the R6a instances, EC2 designed for memory-intensive workloads like SQL and NoSQL databases.. The new instances are built on the AWS Nitro System and are powered by AMD Milan processors.
-
Amazon Redshift Serverless Generally Available to Automatically Scale Data Warehouse
Amazon recently announced the general availability of Redshift Serverless, an elastic option to scale data warehouse capacity. The new service allows data analysts, developers and data scientists to run and scale analytics without provisioning and managing data warehouse clusters.
-
Shopify’s Practical Guidelines from Running Airflow for ML and Data Workflows at Scale
Shopify engineering shared its experience in the company's blog post on how to scale and optimize Apache Airflow for running ML and data workflows. They shared practical solutions for the challenges they faced like slow file access, insufficient control over DAG, irregular level of traffic, resource contention among workloads, and more.
-
Microsoft Announces the General Availability of Its Gateway Load Balancer in All Regions
Gateway Load Balancer is a fully-managed service enabling enterprises to deploy, scale, and enhance the availability of third-party network virtual appliances (NVAs) in Azure. Microsoft recently announced the general availability of Gateway Load Balancer in all public regions, Azure China cloud regions, and Azure Government cloud regions.
-
AWS Announces General Availability of its Cloud WAN for Centralized Workload Management
Recently AWS announced the general availability (GA) of its Cloud WAN solution allowing enterprises to set up and manage a complete WAN environment from a single cloud-based console.
-
VMware vSphere+ and vSAN+ Promise to Bring the Benefits of the Cloud to On-Premises Workloads
Recently announced, VMware vSphere+ and vSAN+ integrate Kubernetes with VMware virtualization technology to help transform on-premises workloads into SaaS-enabled infrastructure and simplify its management and evolution, says VMware.
-
AWS Releases IoT ExpressLink: Cloud-Connectivity Software for Hardware Modules
Amazon recently announced the general availability of AWS IoT ExpressLink. The cloud-connectivity software supports wireless hardware modules to build IoT products that connect with cloud services.
-
AWS Mainframe Modernization Service Now Generally Available
Recently, AWS announced the general availability of their mainframe modernization service, allowing customers to migrate and modernize their on-premises mainframe workloads to a managed and highly available runtime environment on AWS. In addition, the service includes tools and resources to help customers plan and implement migration and modernization.
-
Fitting Presto to Large-Scale Apache Kafka at Uber
The need for ad-hoc real-time data analysis has been growing at Uber. They run a large Apache Kafka deployment and need to analyse data going through the many workflows it supports. Solutions like stream processing and OLAP datastores were deemed unsuitable. An article was published recently detailing why Uber chose Presto for this purpose and what it had to do to make it performant at scale.
-
Cockroach Labs 2022 Cloud Report: AMD Outperforms Intel
Cockroach Labs recently released their annual cloud report which evaluates the performance of AWS, Microsoft Azure and Google Cloud for common OLTP workloads. Differently from the past, this year’s report does not indicate a best overall provider, but concludes that AMD instances outperform Intel ones. ARM instances were not covered in the tests.
-
CNCF Accepts KubeVirt as an Incubating Project
Recently, the CNCF promoted KubeVirt from the sandbox to incubating project level. KubeVirt enables users to run virtual machine workloads on top of Kubernetes in a Kubernetes-native way.
SPONSORED BY
Live Webinar presented by Auth0 |
Q&A moderated by InfoQ
Why Identity Should Form the Foundation of Your Product Strategy
Learn how to choose the most appropriate approach when implementing identity, deployment considerations for new and existing applications and services, how to build on a strong identity foundation, and more.