InfoQ Homepage Big Data Content on InfoQ
-
/articles/covid-social-media-machine-learning/en/smallimage/covid-social-media-machine-learning-s-1601562448288.jpeg)
COVID-19 and Mining Social Media - Enabling Machine Learning Workloads with Big Data
In this article, author Adi Pollock discusses how to enable machine learning workloads with big data to query and analyze COVID-19 tweets to understand social sentiment towards COVID-19.
-
/articles/cloud-to-cloudlets-data-processing/en/smallimage/cloud-to-cloudlets-data-processing-s-1601399656392.jpeg)
From Cloud to Cloudlets: a New Approach to Data Processing?
The growing popularity of small, distributed clouds, or “cloudlets” is an implicit recognition of the limitations of the “traditional” cloud model, and could signal a major shift in the way that data is collected, stored, and processed.
-
/articles/dataops-devops-scale-speed/en/smallimage/combining-dataops-devops-scale-at-speed-logo-small-1597065920955.jpg)
Combining DataOps and DevOps: Scale at Speed
DataOps is an extension of DevOps standards and processes into the data analytics world. It's about streamlining the processes involved in processing, analyzing and deriving value from big data.
-
/articles/data-leadership-book-review-interview/en/smallimage/Data-leadership-s-1595453339836.jpg)
Data Leadership Book Review and Interview
Data Leadership book, authored by Anthony Algmin, covers the data leadership topic and how data leaders should manage and govern the data management programs in their organizations. Data Leadership is how organizations choose to apply their energy and resources toward creating data capabilities to influence their business.
-
/articles/apache-arrow-java/en/smallimage/arrow-s-1590010308987.jpg)
Apache Arrow and Java: Lightning Speed Big Data Transfer
Apache Arrow puts forward a cross-language, cross-platform, columnar in-memory data format for data. It is designed to eliminate the need for data serialization and reduce the overhead of copying.
-
/articles/data-analytics-agility/en/smallimage/data-analytics-agility-logo-1567491976359.jpg)
Data Analytics in the World of Agility
Is it all about customer-centric business, or is there any data left? Can we integrate data analytics and customer empathy? This article explores how we can move towards a more customer-centric business and what information we require in order to understand the most valuable thing we have: our customer.
-
/articles/anamoly-detection-yurita/en/smallimage/logo-1562313713564.jpg)
Stream Processing Anomaly Detection Using Yurita Framework
In this article, author Guy Gerson discusses the stream processing anomaly detection framework they developed by PayPal, called Yurita. The framework is based on Spark Structured Streaming.
-
/articles/data-processing-redis-spark-streaming/en/smallimage/real-time-data-processing-logo-small-1557414118420.jpg)
Real-Time Data Processing Using Redis Streams and Apache Spark Structured Streaming
Structured Streaming, introduced with Apache Spark 2.0, delivers a SQL-like interface for streaming data. Redis Streams enables Redis to consume, hold and distribute streaming data between multiple producers and consumers. In this article, author Roshan Kumar walks us through how to process streaming data in real time using Redis and Apache Spark Streaming technologies.
-
/articles/challenges-of-data-preparation-for-predictive-maintenance/en/smallimage/spark-logo-s-1546543470140.jpeg)
Conquering the Challenges of Data Preparation for Predictive Maintenance
Predictive maintenance (PdM) applications aim to apply machine learning (ML) on IIoT datasets in order to reduce occupational hazards, machine downtime, and other costs. In this article, the author addresses some of the data preparation challenges faced by the industrial practitioners of ML and the solutions for data ingest and feature engineering related to PdM.
-
/articles/analytics-zoo/en/smallimage/analytics-zoo-s-1544204138443.jpeg)
Analytics Zoo: Unified Analytics + AI Platform for Distributed Tensorflow, and BigDL on Apache Spark
In this article we described how Analytics Zoo can help real-world users to build end-to-end deep learning pipelines for big data, including unified pipelines for distributed TensorFlow and Keras on Apache Spark, easy-to-use abstractions such as transfer learning and Spark ML pipeline support, built-in deep learning models and reference use cases, etc.
-
/articles/sentiment-analysis-whats-with-the-tone/en/smallimage/sentiment-analysis-whats-with-the-tone-s-1543275808291.jpeg)
Sentiment Analysis: What's with the Tone?
Sentiment analysis is widely applied in voice of the customer (VOC) applications. In this article, the authors discuss NLP-based Sentiment Analysis based on machine learning (ML) and lexicon-based approaches using KNIME data analysis tools.
-
/articles/spark-application-monitoring-influxdb-grafana/en/smallimage/spark-application-monitoring-influxdb-grafana-s-1542381806762.jpeg)
Spark Application Performance Monitoring Using Uber JVM Profiler, InfluxDB and Grafana
In this article, author Amit Baghel discusses how to monitor the performance of Apache Spark based applications using technologies like Uber JVM Profiler, InfluxDB database and Grafana data visualization tool.