AWS recently announced that Amazon Elastic Container Service (Amazon ECS) supports an integration with Amazon Elastic Block Store (Amazon EBS), which makes it easier for users to run a broader range of data processing workloads.
With the integration, users can provision Amazon EBS storage for their ECS tasks running on AWS Fargate and Amazon Elastic Compute Cloud (Amazon EC2) without needing to manage storage or compute. Moreover, users have various storage options for their containerized applications running on Amazon ECS. By default, Fargate tasks come with 20 GiB of ephemeral storage. Still, users can configure up to 200 GiB for tasks requiring extra storage, such as downloading large container images or temporary scratch work.
In addition, Amazon ECS allows configuring Amazon Elastic File System EFS for applications requiring concurrent access to a shared dataset, suitable for workloads like web applications and machine learning frameworks, while supporting simultaneous attachment to multiple tasks across a region. Alternatively, for applications needing high-performance, cost-effective storage exclusive to individual tasks, Amazon ECS enables the provision and attachment of Amazon EBS storage, known for low-latency, high-performance block storage within an Availability Zone.
Users can leverage the EBS volume integration to their ECS tasks by using the option to set the volume mount point for their container in the task definition and specify Amazon EBS storage requirements for their Amazon ECS task during runtime. In most scenarios, getting started involves merely indicating the required volume size for the task, with the option to configure all EBS volume attributes and the desired file system for formatting the volume.
Create Task Definition in the AWS Console (Source: AWS News blog post)
A respondent, Zenin, questions the storage on a Reddit thread:
Yet, it's not exactly persistent storage, correct? If your container crashes and gets started on a new node, it tosses the EBS volume and makes a new one from scratch, right? So, is this really a way to get a big scratch space or, with a snapshot, a big, predefined data set loaded, but not something to use with, say, a database? Or am I misreading this?
With an AWS employee responding:
That is correct. Currently, the primary use case for EBS volume attachment to ECS is for getting large amounts of data to your task quickly and efficiently. You should find it to be much faster and more performant than trying to stuff a lot of data in a container image, or download it on the fly after the task starts. We know we have more work to do in order to enable truly stateful services, which is why EBS volume reattachment is not yet part of this launch.
The Amazon ECS integration with Amazon EBS is currently available in nine AWS regions: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), and Europe (Stockholm). Costs involved in using the integration depend on the usage of Amazon EBS (volumes and snapshots) – details of pricing are available on the Amazon EBS pricing page and Amazon EBS volumes in ECS in the AWS documentation.