BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News AWS S3 Introduces Account-Regional Namespaces, Ending 18 Years of Global Bucket Name Collisions

AWS S3 Introduces Account-Regional Namespaces, Ending 18 Years of Global Bucket Name Collisions

Listen to this article -  0:00

AWS recently announced account-regional namespaces for S3 general-purpose buckets, addressing an 18-year-old limitation that forced developers to work around global bucket name collisions. Teams can now create buckets with predictable names scoped to their AWS account and region, simplifying infrastructure-as-code templates.

S3's global namespace has been a persistent pain point since 2006. When "mybucket" is taken anywhere in the world, you're stuck with "mybucket-prod-v2-final" or worse. Infrastructure teams resorted to workarounds: deterministic hashes, encrypted project names with decryption scripts, random suffixes in Terraform, and CloudFormation pseudo-random IDs. All just to guarantee bucket creation wouldn't fail with "BucketAlreadyExists" in production.

The account-regional namespace changes the rules. Buckets now follow the format {prefix}-{account-id}-{region}-an, where -an marks the account-regional suffix. AWS account 123456789012 can create "mybucket-123456789012-us-east-1-an" without checking if someone else already claimed it. The 12-digit account ID acts as a natural partition, and other accounts attempting to use that exact suffix get automatically rejected.

(Source: AWS News blog post)

This matters most for infrastructure-as-code. CloudFormation templates can now use the BucketNamePrefix property instead of concatenating pseudo-parameters and hoping for uniqueness:

yaml

BucketNamePrefix: 'amzn-s3-demo-bucket'
BucketNamespace: 'account-regional'

CloudFormation handles the account ID and region automatically. Terraform and Pulumi templates get similar simplifications. Multi-account organizations can enforce consistent naming across hundreds of AWS accounts without fear of collisions.

Security teams gain enforcement capabilities through a new IAM condition key: s3:x-amz-bucket-namespace. Organizations can require all new buckets to use account-regional namespacing via service control policies, preventing teams from falling back on the collision-prone global namespace and making compliance auditing straightforward. Filter for buckets without the -an suffix, and you've found the legacy ones.

The global namespace also posed security risks beyond naming collisions. As one Reddit commenter noted:

The global namespace posed a security risk that enabled confused deputy attacks. While this new enforcement is currently available in most regions, it is not yet active in the Middle East regions. This change helps automate a naming convention that many teams were already using in their infrastructure-as-code to ensure uniqueness.

AWS is catching up to patterns that Azure and Google Cloud implemented from the start. Azure Blob Storage always scoped storage account names to subscriptions, and Google Cloud Storage always scoped buckets to projects. AWS stuck with the global namespace for nearly two decades, now offering the account-scoped alternative as opt-in rather than a breaking change.

There's evidence that developers actively worked around these limitations. A 2024 Hacker News thread showed teams using deterministic hashes and encrypting project names to avoid the global namespace problem. One comment summarized the pattern:

Every time I do AWS now all the bucket names are usually named <project>-<deterministic hash from a seed value>.

The CLI and SDK support is straightforward. The AWS CLI takes a --bucket-namespace account-regional flag, and the Python Boto3 SDK adds a BucketNamespace parameter to create_bucket() calls. The account ID is returned by STS GetCallerIdentity.

Documentation emphasizes this won't break existing deployments. Global namespace buckets continue working as before. The feature exists for teams wanting predictable bucket creation in CI/CD pipelines, multi-account organizations needing consistent naming, or anyone tired of handling BucketAlreadyExists exceptions.

Lastly, the feature is currently available across 35 AWS regions at no additional cost, though it's not yet available in the Middle East (Bahrain) and the Middle East (UAE). Existing buckets can't be renamed; this only applies to new buckets. It's limited to general-purpose buckets; S3 table buckets, directory buckets, and vector buckets already existed in account-level or zonal namespaces.

About the Author

Rate this Article

Adoption
Style

BT