Twitter has open-sourced Storm, its distributed, fault-tolerant, real-time computation system, at GitHub under the Eclipse Public License 1.0. Storm is the real-time processing system developed by BackType, which is now under the Twitter umbrella.
Microsoft wants to give C++ developers tools for writing parallel applications running on zillions of GPUs/cores locally or in the cloud.
With the increased emphasis on multi-core systems an understanding of parallel and concurrent programming is more important than ever. Fortunately .NET 4 has made a lot of advances in the types of synchronization primitives available to developers. One such primitive is the Barrier, which Emad Omara uses to implement a parallel merge sort.
The latest Scala release 2.9.0 introduces parallel collections to easily utilize multicore processors. Other new features are an improved REPL, ScalaDoc and new packages for interacting with the operating system.
TPL Dataflow is Microsoft’s new library for highly concurrent applications. Using asynchronous message passing and pipelining, it promises to offer more control than thread pools and better performance than manual threading. The downside is that you have to adhere to design patterns that may be unfamiliar to .NET programmers.
The new Async CPT for VB and C# looks like it may actually make it into the core language. But with all the emphasis on multi-core systems, why is Microsoft investing so heavily in syntax for designed specifically for making single-threaded asynchronous programming easier?
This month, Yahoo! released a new open source framework for "processing continuous, unbounded streams of data." The framework, named S4, allows for massively distributed computations over data that is constantly changing. InfoQ examines some of the examples and compares S4 to other technologies.
In a recent blog post the Visual Basic team let slip an announcement that Visual Basic and C# would be getting a new syntax for asynchronous programming. Built on top of the Task Parallel Library that was introduced in .NET 4, this adds the Async and Await keywords to both languages.
Brian Goetz and Cliff Click spoke at JavaOne conference last week about concurrency revolution from a hardware perspective. They said CPU designers will focus on parallelism in the future for increasing throughput of the systems. They also discussed some point solutions like Thread Pools, Fork/Join, Map/Reduce and Actors to achieve the concurrency in applications.
InfoQ catches up with Gary Frost from AMD who unveiled an alpha release of Aparapi, an API that allows programmers to write logic in Java to be executed on a GPU. GPUs are the massively parallel hardware acceleration chips originally installed in PCs to boost graphics rendering performance but that are now pushed to other kinds of compute-intensive tasks that have nothing to do with graphics.
The patterns&practices team has released Parallel Programming with Microsoft .NET, a book containing guidance for writing parallel programs for .NET. In essence the book contains 6 design patterns for parallel programming accompanied by code samples.
Optional parameters have always been part of .NET, but with C# unwilling to support it, using them was generally considered taboo unless work with COM libraries. Now that C# 4 does support them, we are starting to see them used for a lot more than just legacy code. Other uses include interoperability with dynamic languages, immutable data structures, and various parts of ASP.NET MVC.
Brahma is an open source C# library that provides support for parallel computations running on a variety of processors. Currently, Brahma has a GPU provider but its modular structure allows using different providers for other types of processors. One C# method can contain both statements running on CPU and GPU without additional glue code.