InfoQ Homepage S3 Content on InfoQ
-
S3 Compatibility Doesn't Guarantee S3-Level Security
Security researchers at Wiz recently examined S3-compatible object storage services across six popular neoclouds, revealing significant security gaps compared to Amazon S3. While S3 has become the de facto standard for object storage, most services lack several of AWS's security protections.
-
Canva Shares S3 Based Architecture for Session Revocation across Hundreds of Millions of Sessions
Canva redesigned session revocation infrastructure to support 100M active sessions while reducing database lookups. The architecture uses Amazon S3 for durable revocation records and distributes compact, in-memory indexes to application gateways. Canva said the design improved deployment speed, reduced database infrastructure requirements, and cut the revocation cache memory footprint by 87.5%.
-
AWS Lambda's Self-Managed Code Storage Lifts the Account Quota, Not the Function Size Limit
AWS Lambda can now reference deployment packages directly in customer-owned S3 buckets, removing the per-Region code storage quota and raising the managed default from 75 GB to 300 GB. Per-function package limits are unchanged, and UpdateFunctionCode is still required after replacing an object. Terraform provider support remains an open enhancement request.
-
Airbnb Shares Architecture behind Sitar-Agent Dynamic Configuration Sidecar for Kubernetes Services
Airbnb engineers detailed Sitar-agent, a Kubernetes sidecar for dynamic configuration delivery across tens of thousands of pods, processing updates several times per minute. The system was redesigned with Java, Amazon S3 snapshot bootstrapping, and a migration from Sparkey to SQLite to improve reliability, startup performance, and configuration availability at scale.
-
AWS Introduces Amazon S3 Annotations
AWS recently announced Amazon S3 Annotations, a feature that lets teams attach rich, searchable context such as summaries, classifications, compliance data, or AI-generated insights directly to S3 objects. Annotations can be updated independently of the object and queried across datasets, reducing the need for separate metadata systems.
-
AWS Introduces S3 Files, Bringing File System Access to S3 Buckets
AWS recently introduced S3 Files, which lets users mount an Amazon S3 bucket and access its data through a standard file system interface. Applications can read and write files using standard file operations, while the system automatically translates them into S3 requests, allowing compute services to work directly with data stored in S3.
-
Inside Agoda’s Storefront: a Latency-Aware Reverse Proxy for Improving DNS Based Load Distribution
Agoda engineers developed Storefront, a Rust-based S3-compatible reverse proxy that improves load balancing, request routing, and observability across large-scale object storage systems. The proxy addresses DNS-based distribution limitations, implements latency-aware routing, cross-data-center optimizations, IO safeguards, credential-less authentication, and exposes telemetry via OpenTelemetry.
-
AWS S3 Introduces Account-Regional Namespaces, Ending 18 Years of Global Bucket Name Collisions
AWS introduced account-regional namespaces for S3, fixing global bucket name collisions that broke IaC automation for 18 years. The new format is {prefix}-{account-id}-{region}-an. CloudFormation gets the BucketNamePrefix property, and IAM gets the s3:x-amz-bucket-namespace condition key. This prevents confused-deputy attacks by making names unpredictable when there is no account ID.
-
Pinterest’s CDC-Powered Ingestion Slashes Database Latency from 24 Hours to 15 Minutes
Pinterest launched a next-generation CDC-based database ingestion framework using Kafka, Flink, Spark, and Iceberg. The system reduces data availability latency from 24+ hours to 15 minutes, processes only changed records, supports incremental updates and deletions, and scales to petabyte-level data across thousands of pipelines, optimizing cost and efficiency.
-
Cloudflare Introduces Local Uploads for R2 to Cut Cross-Region Write Latency by 75%
Cloudflare has recently introduced Local Uploads for R2 in open beta. The new feature optimizes write performance for globally distributed users without changing bucket location, reducing cross-region write latency.
-
AWS Adds Intelligent-Tiering and Replication for S3 Tables
AWS has introduced Intelligent-Tiering and cross-region replication for S3 Tables to automate cost optimization and data availability for Apache Iceberg workloads. These features allow data to transition to lower-cost storage tiers based on access patterns while maintaining consistent, read-only table replicas across regions and accounts without manual synchronization.
-
DuckDB's WebAssembly Client Allows Querying Iceberg Datasets in the Browser
DuckDB has recently introduced end-to-end interaction with Iceberg REST Catalogs directly within a browser tab, requiring no infrastructure setup. The new feature leverages DuckDB-Wasm, a WebAssembly port of DuckDB that runs in the browser, allowing users to query, read, and write Iceberg tables in a serverless manner.
-
Amazon S3 Vectors Reaches GA, Introducing "Storage-First" Architecture for RAG
AWS has announced the general availability of Amazon S3 Vectors, increasing per-index capacity forty-fold to 2 billion vectors. By natively integrating vector search into the S3 storage engine, the service introduces a "Storage-First" architecture that decouples compute from storage, reducing total cost of ownership by up to 90% for large-scale RAG workloads.
-
MinIO GitHub Repository in Maintenance Mode: What's Next for the Open Source Object Storage?
After a contentious license change and the removal of administrator functionalities from the console, the company behind the popular open-source object storage server Minio recently announced that the project will now enter maintenance mode. The change has raised discussion in the community about the need for a fork, the challenges of open source projects, and the current alternatives.
-
Yelp Publishes Blueprint for Managing S3 Server-Access Logs at Massive Scale
In a detailed engineering post, Yelp shared how it built a scalable and cost-efficient pipeline for processing Amazon S3 server-access logs (SAL) across its infrastructure, overcoming traditional limitations of raw log storage and querying at high volume.