VMware Infrastructure 3 Book Excerpt and Author Interview
VMware Infrastructure 3: Advanced Technical Design Guide and Advanced Operations Guide provides a wealth of practical insights into setting up virtualization in todays corporate environments.
Tracking change and innovation in the enterprise software development community
Posted by Scott Delap on Nov 10, 2006 09:01 AM
Clustered grid computing software does not simply happen. Efficient architectures must be designed. One of the core technologies used by Google is the MapReduce programming model which allows for the processing and generation of large data sets. By defining a scalable program structure upfront Map Reduce allows algorithms to easily scale across machines:
Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google's clusters every day.
Doug Cutting the creator of Lucene and now an employee of Yahoo has been working on an open source implementation of MapReduce and called Hadoop written in Java which also includes a distributed file system. Hadoop has already been tested on clusters up to 600 nodes.
Hadoop is a framework for running applications on large clusters of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named map/reduce, where the application is divided into many small fragments of work, each of which may be executed or reexecuted on any node in the cluster. In addition, it provides a distributed file system that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both map/reduce and the distributed file system are designed so that node failures are automatically handled by the framework.
Amazon recently released their EC2 Elastic Computing cloud which allows developers to acquisition computing power a the rate of $0.10 per hour consumed. Recently work has been done to allow Hadoop to run on EC2. This combination will allow developers to write scalable algorithms and then bring up large numbers of servers for computing power which can then be then shut them down when they are not needed.
Hibernate without Database Bottlenecks
WebSphere Virtual Enterprise 3 minute demo
Introducing application infrastructure virtualization and WebSphere Virtual Enterprise
Rainmaking - IBM's software virtualization strategy (Jerry Cuomo CTO blog)
Guide to Calculating ROI with Terracotta Open Source JVM Clustering
VMware Infrastructure 3: Advanced Technical Design Guide and Advanced Operations Guide provides a wealth of practical insights into setting up virtualization in todays corporate environments.
Can a system that is so large it cannot be comprehended be "designed" in a conventional sense? The foundations of computing are about to change. In this talk, Richard P. Gabriel explores why and how.
Ruby 1.9's Fibers and non-blocking I/O are getting more attention - we talked to Mohammad A. Ali of the NeverBlock project and Tony Arcieri of the Revactor project.
Tim Mackinnon talks about the aspirations behind the Agile principles and practices, the desire to become efficient, to write quality code which does not end up being thrown away.
Brian Goetz discusses the difficulties of creating multithreaded programs correctly, incorrect synchronization, race conditions, deadlock, STM, concurrency, alternatives to threads, Erlang, Scala.
Often the hardest part of changing technologies is language syntax differences. This new article provides Java developers with a transition guide to Actionscript which forms the foundation of Flex.
Neal Ford talks about having multiple languages running on one of the two major platforms: Java and .NET. He also presents the advantages offered by Ruby compared to static languages like Java or C#.
David Anderson talks about the history of Agile, the current status of it and his vision for the future. The role of Agile consists in finding ways to implement its principles.
1 comment
Reply