InfoQ Homepage S3 Content on InfoQ
-
Pfizer Uses Serverless Architecture on AWS to Scale Processing of Digital Biomarkers
Pfizer upgraded the serverless architecture for processing digital biomarker data at scale to make it more flexible and configurable. They created a framework that uses a file processing pipeline built with AWS Step Functions and other serverless services, as well as a custom Python package for data ingestion and processing.
-
AWS Launches Amazon S3 Dual-Layer Server-Side Encryption with Keys Stored in AWS KMS
Recently AWS launched Amazon S3 dual-layer server-side encryption with keys stored in AWS Key Management Service (DSSE-KMS), a new encryption option in Amazon S3 that applies two layers of encryption to objects when they are uploaded to an Amazon Simple Storage Service (Amazon S3) bucket.
-
Datadog Creates Scalable Data Ingestion Architecture
Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs.
-
AWS Introduces Athena Provisioned Capacity
AWS recently announced a new feature Provisioned Capacity for Athena, that allows users to run SQL queries on fully-managed compute capacity for a fixed price and no long-term commitments.
-
AWS Announces Open Source Mountpoint for Amazon S3
During the latest Pi Day, AWS announced Mountpoint for Amazon S3, an open-source file client to deliver high throughput access on Amazon S3. Currently in alpha, the local mount point provides high single-instance transfer rates and is primarily intended for data lake applications.
-
Amazon S3 Encrypts All New Objects with AES-256
Since January 5th, Amazon S3 encrypts all new objects by default with AES-256 to protect data at rest. S3 automatically applies server-side encryption using Amazon S3-managed keys for each new object, unless a different encryption option is specified.
-
AWS Announces Upcoming Security Changes in April 2023 for Amazon S3
Recently AWS announced it would make two changes to Amazon Simple Storage Service (Amazon S3): all buckets in a region will have S3 Block Public Access enabled and access control lists (ACLs) disabled by default. These changes will take effect in April 2023 and will be rolled out by the company in all AWS regions within weeks.
-
Recap of AWS re:Invent 2022
After a virtual-only event in 2020 and a reduced-size 2021 edition, re:Invent was back last week in Las Vegas with over 50,000 attendees for the 11th edition. During multiple sessions and keynotes at the largest AWS yearly conference, the cloud provider announced new services and features, with the focus more on business solutions and data options than new building blocks.
-
Cloudflare Introduces Super Slurper Feature for Its R2 Object Storage in Private Beta
Cloudflare recently announced the R2 Super Sluper feature, which enables developers to move all their data to R2 in one giant slurp or sip by sip through an intuitive UI and API. The feature is in private beta by signing up.
-
Open Source Skyplane Targets Faster and Cheaper Data Transfers between Clouds
A team of researchers at the RISELab at UC Berkeley has recently released Skyplane, an open source tool to optimize the transfer of large datasets between cloud providers, reducing transfer times and costs.
-
Amazon File Cache Now Generally Available
Amazon recently announced the general availability of File Cache, a managed high-speed cache for processing file data stored in disparate locations. The new service can be linked to multiple sources including on premises network file systems and managed AWS services like Amazon FSx or S3.
-
Cloudflare R2 Storage Generally Available
Cloudflare recently announced the general availability of R2 storage, an S3-compatible object storage without egress charges. The new service provides dynamic functionalities integrating with Cloudflare Workers.
-
Azure Data Explorer Supports Native Ingestion from Amazon S3
Microsoft recently announced the ability to natively ingest data from Amazon S3 into Azure Data Explorer (ADX). The new feature simplifies multi-cloud data analytics deployments, bringing data from Amazon S3 to Azure, without relying on custom ETL pipelines.
-
AWS Backup for Amazon S3 Generally Available
Amazon recently announced the general availability of AWS Backup for Amazon S3, a managed option to centralize data protection for applications storing data on S3. The new feature automates the backup and restore of application data on S3 and combines it with the backup of other services, including instances and databases.
-
Amazon S3 Supports New Checksum Algorithms for Integrity Checking
Amazon S3 recently introduced support of four checksum algorithms for data integrity checking on upload and download requests. Amazon claims that the enhancements to the AWS SDK and S3 API accelerates integrity checking of the S3 requests by up to 90%.