InfoQ Homepage S3 Content on InfoQ
-
Running Apache Flink Applications on AWS KDA: Lessons Learnt at Deliveroo
Deliveroo introduced Apache Flink into its technology stack for enriching and merging events consumed from Apache Kafka or Kinesis Streams. The company opted to use AWS Kinesis Data Analytics (KDA) service to manage Apache Flink clusters on AWS and shared its experiences from running Flink applications on KDA.
-
Inside InfluxDB 3.0: Exploring InfluxDB’s Scalable and Decoupled Architecture
InfluxData recently unveiled the system architecture for InfluxDB 3.0, its newest time-series DB. Its architecture encompasses four major components responsible for data ingestion, querying, compaction, and garbage collection and includes two main storage types. The architecture caters to operating the DB on-premise and natively on major cloud providers.
-
Amazon Introduces AWS HealthImaging to Store and Analyze Medical Imaging Data
At the recent AWS Summit in New York, Amazon announced AWS HealthImaging. The new HIPAA-eligible service helps healthcare providers to store, analyze, and share medical imaging data at scale.
-
Pfizer Uses Serverless Architecture on AWS to Scale Processing of Digital Biomarkers
Pfizer upgraded the serverless architecture for processing digital biomarker data at scale to make it more flexible and configurable. They created a framework that uses a file processing pipeline built with AWS Step Functions and other serverless services, as well as a custom Python package for data ingestion and processing.
-
AWS Launches Amazon S3 Dual-Layer Server-Side Encryption with Keys Stored in AWS KMS
Recently AWS launched Amazon S3 dual-layer server-side encryption with keys stored in AWS Key Management Service (DSSE-KMS), a new encryption option in Amazon S3 that applies two layers of encryption to objects when they are uploaded to an Amazon Simple Storage Service (Amazon S3) bucket.
-
Datadog Creates Scalable Data Ingestion Architecture
Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs.
-
AWS Introduces Athena Provisioned Capacity
AWS recently announced a new feature Provisioned Capacity for Athena, that allows users to run SQL queries on fully-managed compute capacity for a fixed price and no long-term commitments.
-
AWS Announces Open Source Mountpoint for Amazon S3
During the latest Pi Day, AWS announced Mountpoint for Amazon S3, an open-source file client to deliver high throughput access on Amazon S3. Currently in alpha, the local mount point provides high single-instance transfer rates and is primarily intended for data lake applications.
-
Amazon S3 Encrypts All New Objects with AES-256
Since January 5th, Amazon S3 encrypts all new objects by default with AES-256 to protect data at rest. S3 automatically applies server-side encryption using Amazon S3-managed keys for each new object, unless a different encryption option is specified.
-
AWS Announces Upcoming Security Changes in April 2023 for Amazon S3
Recently AWS announced it would make two changes to Amazon Simple Storage Service (Amazon S3): all buckets in a region will have S3 Block Public Access enabled and access control lists (ACLs) disabled by default. These changes will take effect in April 2023 and will be rolled out by the company in all AWS regions within weeks.
-
Recap of AWS re:Invent 2022
After a virtual-only event in 2020 and a reduced-size 2021 edition, re:Invent was back last week in Las Vegas with over 50,000 attendees for the 11th edition. During multiple sessions and keynotes at the largest AWS yearly conference, the cloud provider announced new services and features, with the focus more on business solutions and data options than new building blocks.
-
Cloudflare Introduces Super Slurper Feature for Its R2 Object Storage in Private Beta
Cloudflare recently announced the R2 Super Sluper feature, which enables developers to move all their data to R2 in one giant slurp or sip by sip through an intuitive UI and API. The feature is in private beta by signing up.
-
Open Source Skyplane Targets Faster and Cheaper Data Transfers between Clouds
A team of researchers at the RISELab at UC Berkeley has recently released Skyplane, an open source tool to optimize the transfer of large datasets between cloud providers, reducing transfer times and costs.
-
Amazon File Cache Now Generally Available
Amazon recently announced the general availability of File Cache, a managed high-speed cache for processing file data stored in disparate locations. The new service can be linked to multiple sources including on premises network file systems and managed AWS services like Amazon FSx or S3.
-
Cloudflare R2 Storage Generally Available
Cloudflare recently announced the general availability of R2 storage, an S3-compatible object storage without egress charges. The new service provides dynamic functionalities integrating with Cloudflare Workers.