A surprisingly common theme at the Splunk Conference is the architectural question, “Should I push, pull, or search in place?”
If you could handle all of the data you need to work with on one machine, then there is no reason to use big data techniques. So clustering is pretty much assumed for any installation larger than a basic proof of concept. In Splunk Enterprise, the most common type of cluster you’ll be dealing with is the Indexer Cluster.
When working with Hadoop, with or without Hunk, there are a number of ways you can accidentally kill performance. While some of the fixes require more hardware, sometimes the problems can be solved simply by changing the way you name your files.
Splunk is jumping into the service-monitoring sector with a new visualization called IT Service Intelligence.
Splunk can now store archived indexes on Hadoop. At the cost of performance, this offers a 75% reduction in storage costs without losing the ability to search the data. And with the new adapters, Hadoop tools such as Hive and Pig can process the Splunk-formatted data.
Splunk opened their big data conference with an emphasis on “making machine data accessible, usable, and valuable to everyone”. This is a shift from their original focus: indexing arbitrary big data sources. Reasonably happy with their ability to process data, they want to ensure that developers, IT staff, and normal people have a way to actually use all of the data their company is collecting.
After a 5-year stint, Kellan Elliott-McCrea, left Etsy. Elliott-McCrea was Etsy's CTO for the past 4 years and VP of Engineering before that. During those five years both the software product and the engineering team underwent radical changes. In the article announcing his departure, Elliott-McCrea expounds five theories that guided him through those changes.
On August 12, Google announced that its big data processing service has reached general availability. This managed service allows customers to build pipelines that manipulate data prior to being processed by big data solutions. Cloud Dataflow supports both streaming and batch programming in a unified model.
IBM has announced LinuxONE, a Linux-only hardware portfolio which runs SUSE, Red Hat or Ubuntu distributions and adds support for different open-source tools such as Docker and Chef. This offering is targeted to both large enterprises and mid-size businesses.
James Governor from RedMonk has written about how immutable infrastructure approaches are applicable to microservices. In his view, all microservices must be immutable and developers will observe the same benefits which others are already seeing in lower layers of the software stack.
It is now possible to run Windows-specific software inside containers. Windows Server 2016 Technical Preview 3 is the first Windows release to support this technology. Windows Server Containers offer very similar capabilities to Linux containers, but in the context of the Windows ecosystem. Docker also released its own technical preview that showcases how it's able to manage Windows containers.
Oracle shocked the Java world this week by announcing the dismissal of some of their top Java evangelists including Cameron Purdy and Simon Ritter.
Airbnb recently opensourced Airflow, its own data workflow management framework. Airflow is being used internally at Airbnb to build, monitor and adjust data pipelines. Airflow’s creator, Maxime Beauchemin and Agari’s Data Architect and one of the framework’s early adopters Siddharth Anand discuss about Airflow, where it can be of use and future plans.
At the fifth ‘Agile on the Beach’ conference, held in Cornwall, UK, several leading practitioners of agile software delivery presented the state-of-the-art and emerging trends within this domain. Key messages included the need for the more rigorous use of the scientific method throughout the software delivery lifecycle, and the benefits provided by applying agile principles to product development.
The AXA Digital Agency deploys the Lean Startup approach, using design thinking, minimum viable product development and growth hacking, to innovate and support the digital transformation at AXA. An interview with Yves Caseau about the importance of innovation, adopting a lean startup approach, learnings from minimum viable products and growth hacking and advice for starting an innovation journey.