At their annual re:Invent conference in Las Vegas, AWS unleashed a flurry of announcements about upcoming cloud services. Amazon outlined over two dozen new capabilities coming to the public cloud, including directly querying data in S3 object storage, building code as part of deployment pipelines, provisioning cheap virtual private servers, and moving data in bulk, ETL-style.
After a brief beta period that saw customers migrate more than 1,000 on-premises databases to the cloud, AWS formally released their Database Migration Service. This on-demand cloud service supports live migration scenarios, and customers who wish to switch their database platform as part of the migration can do so, thanks to a free schema conversion tool.
Modern software increasingly operates on data in near real-time. There is business value in sub-second responses to changing information and stream processing is one way to help turn data into knowledge as fast as possible, Kevin Webber explains in an introduction to Reactive Streams.
On August 12, Google announced that its big data processing service has reached general availability. This managed service allows customers to build pipelines that manipulate data prior to being processed by big data solutions. Cloud Dataflow supports both streaming and batch programming in a unified model.
This post talks about using an agile implementation for data warehouse projects.
Microsoft recently released SvcPerf which is an end to end ETW based trace viewer for manifest based traces.
The AWS team has announced a limited preview of Amazon Redshift, a cloud-hosted data warehouse whose cost and capabilities are poised to disrupt the industry. In addition, AWS revealed two new massive compute instance types, and a data integration tool called Data Pipeline.
Fred Cummins, an EDS fellow, and SOA veteran, wrote an essay last week on "Data Management for SOA". He is looking at how some of the key tenets of service design ("loose coupling" and "autonomy") relate to enterprise data in the context of achieving reuse and enabling change.
ActiveWarehouse, is a significant new plugin that makes it easier to build data warehouses in Rails.