When working with Hadoop, with or without Hunk, there are a number of ways you can accidentally kill performance. While some of the fixes require more hardware, sometimes the problems can be solved simply by changing the way you name your files.
Splunk can now store archived indexes on Hadoop. At the cost of performance, this offers a 75% reduction in storage costs without losing the ability to search the data. And with the new adapters, Hadoop tools such as Hive and Pig can process the Splunk-formatted data.
Splunk opened their big data conference with an emphasis on “making machine data accessible, usable, and valuable to everyone”. This is a shift from their original focus: indexing arbitrary big data sources. Reasonably happy with their ability to process data, they want to ensure that developers, IT staff, and normal people have a way to actually use all of the data their company is collecting.
Twitter has open sourced Diffy, an automated testing tool used in production for discovering potential bugs in new code running on Apache Trift and other HTTP-based services.
Version 3 of WebSharper, the F# framework for developing web applications hits RTM this year. We decided to catch up with Adam Granicz, CEO of IntelliFactory, to learn what new features and improvements WebSharper 3 brings.
James Governor from RedMonk has written about how immutable infrastructure approaches are applicable to microservices. In his view, all microservices must be immutable and developers will observe the same benefits which others are already seeing in lower layers of the software stack.
At the fifth ‘Agile on the Beach’ conference, held in Cornwall, UK, several leading practitioners of agile software delivery presented the state-of-the-art and emerging trends within this domain. Key messages included the need for the more rigorous use of the scientific method throughout the software delivery lifecycle, and the benefits provided by applying agile principles to product development.
At the Golang UK Conference, Peter Bourgon introduced ‘Go kit’, an open source microservice toolkit that can be used to facilitate and standardise the creation of Go-based services within the modern enterprise application stack.
Command Query Responsibility Segregation (CQRS) is the starting point of a change that will have a profound impact on system architecture, Dino Esposito claims in three articles in MSDN Magazine. It’s the first step in an evolution transitioning software architects from the idea of “models-to-persist” to the idea of “events-to-log” and about event-based data instead of data snapshots.
Software architecture is a process; a sequence of strategic design decisions mapping specification and business goals to architecture design, and a thing; a set of views produced by the process that address different stakeholders, Michael Stal states describing how to define a software architecture.
After failing with two proof of concept, mainly with scalability issues, when trying to renew their main website the business decided to take a more agile and incremental approach and in a restart of the project inspired by Domain-Driven Design (DDD) having developers talk with domain experts, Chris Patuzzo explains describing the principles of DDD in the context of a real project.
Preparing for problems like partial failure is the best thing you can do when working with distributed systems, Vaughn Vernon explains in a conversation with InfoQ and refers to a blog post by Jeff Hodges noting its down-to-earth approach and practical advices e.g. designing for partial availability, and using capped exponential back off to restore full operation when dependencies are unavailable.
Redfish 1.0 is defined as a standard and a RESTful API for the management of scale-out commodity servers. Although it was created with the current needs of scalable architectures in mind, Redfish can be used for the management or the integration of the older platforms and their tool chains.
GameAnalytics, maker of a free analytics platform, has recently open sourced gascheduler an Erlang library that provides a generic scheduler for parallel execution of distributed tasks. InfoQ has spoken to Chris de Vries, one of gascheduler’s creators.
Intel has introduced a new feature for its Integrated Native Development Experience (INDE) called Multi-OS Engine that aims to make it easier for Java developers to port their Android apps to the iOS platform.