InfoQ Homepage Hadoop Content on InfoQ
-
Docker Data Science Pipeline
Lennard Cornelis explains why they chose OpenShift and Docker to connect to the Hadoop environment, also how to set up a Docker container running a data science model using Hive, Python, and Spark.
/presentations/ing-docker-hadoop/en/mediumimage/Lenbig-1558321090152.JPG)
-
Scaling Marketplaces at Thumbtack
Nate Kupp shares some of Thumbtack’s key learnings on their journey to scale and their future with fully-managed systems.
/presentations/thumbtack-scalability/en/mediumimage/Natbig-1521080309385.JPG)
-
Best Trade-off Point Algorithm for Efficient Resource Provisioning in Hadoop
Peter Nghiem presents the Best Trade-off Point method and algorithm with mathematical formulas for obtaining the exact optimal number of task resources for any workload running on Hadoop.
/presentations/best-trade-off-point-hadoop/en/mediumimage/Petbig-1509242902336.JPG)
-
Data Science in the Cloud @StitchFix
Stefan Krawczyk discusses how StitchFix used the cloud to enable over 80 data scientists to be productive and have easy access, covering prototyping, algorithms used, keeping schema in sync, etc.
/presentations/stitchfix-cloud/en/mediumimage/STebig.JPG)
-
Streaming Live Data and the Hadoop Ecosystem
Oleg Zhurakousky discusses the Hadoop ecosystem – Hadoop, HDFS, Yarn-, and how projects such as Hive, Atlas, NiFi interact and integrate to support the variety of data used for analytics.
/presentations/streaming-hadoop/en/mediumimage/Olebig.JPG)
-
Achieving Mega-Scale Business Intelligence through Speed of Thought Analytics on Hadoop
Ian Fyfe discusses the different options for implementing speed-of-thought business analytics and machine learning tools directly on top of Hadoop.
/presentations/business-analytics-machine-learning-hadoop/en/mediumimage/Ianbig.JPG)
-
The Mechanics of Testing Large Data Pipelines
Mathieu Bastian explores the mechanics of unit, integration, data and performance testing for large, complex data workflows, along with the tools for Hadoop, Pig and Spark.
/presentations/patterns-data-workflows/en/mediumimage/Matbig.JPG)
-
Hadoop Workflows and Distributed YARN Apps using Spring Technologies
The authors discuss how Spring for Apache Hadoop can make developing workflows with Map Reduce, Spark, Hive and Pig jobs easier, and using Spring Cloud to build distributed apps for YARN.
/presentations/spring-hadoop-workflow-yarn/en/mediumimage/doibig.jpg)
-
Federated Queries with HAWQ - SQL on Hadoop and Beyond
Christian Tzolov shows different integration approaches between HAWQ and GemFire, showing using Spring XD to ingest GemFire data into HDFS and using Spring Boot to implement a RESTful proxy for HAWQ.
/presentations/federated-queries-hawq/en/mediumimage/Chrisbig.JPG)
-
How 30 Years of Ticket Transaction Data Helps you Discover New Shows!
Vaclav Petricek discusses how to train models, architect and build a scalable system powered by Storm, Hadoop, Spark, Spring Boot and Vowpal Wabbit that meets SLAs measured in tens of milliseconds.
/presentations/storm-hadoop-spark/en/mediumimage/Vacbig.JPG)
-
Lightning Fast Cluster Computing with Spark and Cassandra
Piotr Kołaczkowski discusses how they integrated Spark with Cassandra, how it was done, how it works in practice and why it is better than using a Hadoop intermediate layer.
/presentations/spark-cassandra/en/mediumimage/Piobig.jpg)
-
Better Together - Using Spark and Redshift to Combine Your Data with Public Datasets
Eugene Mandel discusses challenges of conforming data sources and compares processing stacks: Hadoop+Redshift vs Spark, showing how the technology drives the way the problem is modeled.
/presentations/hadoop-redshift-spark/en/mediumimage/Eugenebig.jpg)