It's one thing having an in-house training program. But there is a certain deeper insight that can be gained from attending a Q-Con conference in person. Which may be one reason why attendance at the globe-hopping event continues to grow.
Mitchell Hashimoto, creator of Vagrant, gave a talk last month at Velocity Conf London about his vision for a “FutureOps” with immutable infrastructures and built-in failure recovery.
Arun Kejariwal, from Twitter, talked at Velocity Conf London last month about forecasting algorithms used at Twitter to proactively predict system resource needs as well as business metrics such as number of users or tweets. Given the dynamic nature of their data stream, they found that a refined ARIMA model works well once data is cleansed, including removal of outliers.
Trifacta, a data analysis services platform, recently received VC investment to advance on their efforts of making data wrangling easier for data analysts. The goal is to collect, cleanse and munge data in a fraction of the time and effort it currently takes.
Chaitanya Mishra, from Facebook, spoke at Velocity Conf London last month about the approach to scale Facebook’s Android app from a web view interface to a full-fledged native app. To achieve this transition each product team took ownership of their features on Android. A core integration team regression tests and focuses on global app optimization over individual features optimization.
Qubole, a managed Hadoop-as-a-Service offering is now available on Google Compute Engine (GCE). Qubole was so far only available on Amazon's AWS and this announcement follows only a few days after Google releasing GCE into general availability.
The MapReduce paradigm is not always ideal when dealing with large computationally intensive algorithms. A small team of entrepreneurs is building a product called ParallelX to solve that bottleneck by harnessing the power of GPUs to give Hadoop jobs a significant boost.
Mirage OS is a ‘cloud operating system’ that seeks to avoid security vulnerabilities and bloat by facilitating the creation of single purpose virtual appliances. Applications are developed in the OCaml functional programming language and compiled into standalone ‘unikernels’ that run directly on the Xen hypervisor.
As mobile applications are becoming a more central part of companies' IT strategies, testing and analyzing those applications becomes more important. Whereas functional testing of code is part of every project, analyzing behavior and conversion rates is still very new to the mobile sector. Splitforce offers tools and services for application developers to instrument apps for in-app analysis.
Want to do DevOps automation in a Microsoft world? Typically that meant using Microsoft-provided tools like PowerShell and System Center instead of the popular open source tools that have been slow to support the full Microsoft product stack. That’s beginning to change as developers and system administrators can now use tools like Puppet to provision and manage resources in Windows Azure.
EC2 users can now automate the deployment of Apache Mesos, an open-source tool to share cluster resources between multiple data processing frameworks, at scale through a new web service called Elastic Mesos provided by Big Data startup Mesosphere.
The Windows PowerShell Blog has been writing about PowerShell's Desired State Configuration (DSC). DSC is the Microsoft's proposal for computer's configuration management on the Windows world. One of the last articles explains how DSC allows both push and pull configuration modes, two approaches with different pros and cons.
Registration is now open for QCon New York 2014 (Jun 9-13). The third annual conference will feature practitioner-led sessions and case studies on topics like Continuous Delivery, Data Science, HTML5, Agile, Architecture, and more. Register before Dec 21st and save $800.
Jeff Magnusson from Netflix team gave a presentation at QCon SF 2013 Conference about their Data Platform as a Service. Following up to this presentation, we will look at the technology stack and how it helps Netflix to tackle important business decisions.
With Facebook recently releasing Presto as open source, the already crowded SQL-in-Hadoop market just became a tad more intricate. A number of open source tools are competing for the attention of developers: Hortonworks Stinger initiative around Hive, Apache Drill, Apache Tajo, Cloudera’s Impala, Salesforce’s Phoenix (for HBase) and now Facebook’s Presto.