Recently, Amazon announced the general availability of UltraWarm for its Elasticsearch Service on AWS. Ultrawarm is a low cost warm storage tier, and extension to the Elasticsearch Service offering up to three petabytes of storage, at ~90% cost reduction over existing options.
Amazon Elasticsearch Service is a fully managed service that makes it easy for customers to deploy, secure, and run Elasticsearch at scale. And with Ultrawarm, the service now supports two storage tiers, hot and UltraWarm. The first is suitable for indexing, updating, and providing the fastest access to data – while UltraWarm complements the hot tier to add support for high volumes of older, less-frequently accessed data to enable customers to take advantage of a lower storage cost.
Martin Beeby, a principal advocate for Amazon Web Services, stated in a blog post on the GA of UltraWarm:
There are many use cases for the Amazon Elasticsearch Service, from building a search system for your website, storing, and analyzing data from application or infrastructure logs. We think this new storage tier will work particularly well for customers that have large volumes of log data.
Users can leverage UltraWarm when setting up a new domain in the Elasticsearch service using the AWS Management Console. During the workflow to set up a new cluster, users can in the configuration step, check the box to Enable UltraWarm data nodes and select the Instance type they want to use. Furthermore, when selecting the instance type, the number of warm nodes has to be at least two (minimum). Note that users can also use the CLI, or SDK to create a new domain or update an existing one.
Source: https://aws.amazon.com/blogs/aws/general-availability-of-ultrawarm-for-amazon-elasticsearch-service/
UltraWarm nodes in the Elasticsearch Service use Amazon S3 and a sophisticated caching solution to improve performance. Under the hood, the UltraWarm determines which blocks of data are accessed less frequently, and which subsequently are moved to S3 while it caches the more frequent data. Moreover, this process increases query execution and lowers the costs of storage.
Raju Gulabani, VP of databases and analytics at AWS, said in a press release:
Our customers tell us that log data offers a wealth of operational and security insights, but that the storage of log data quickly adds up, and proves cost-prohibitive over the medium and long term. UltraWarm is the most cost-effective Elasticsearch-compatible storage solution available. It is also performance-optimized, so customers can investigate and interactively visualize their data while they embrace data at scale.
Also, an Italian AWS Advanced Consulting Partner, beSharp, stated in a tweet:
This is great for reducing costs for very large clusters, especially if big time frames are used for #Elasticsearch service. Also, storage and computational power can scale separately!
Lastly, UltraWarm supports Elasticsearch application programming interfaces (APIs), tools such as Kibana to visualize stored data, and features, including enterprise-grade security with fine-grained access control, encryption at rest and in flight, integrated alerting and SQL querying.
Currently, Amazon Elasticsearch Service UltraWarm is available for domains using Elasticsearch version 6.8 and higher across 22 AWS Regions globally. Furthermore, pricing details for AWS Elasticsearch, including UltraWarm, are available on the pricing page.