InfoQ Homepage Big Data Content on InfoQ
-
/articles/deep-learning-apache-spark-nvidia-gpu/en/smallimage/spark-logo-1623166971929.jpg)
Accelerating Deep Learning on the JVM with Apache Spark and NVIDIA GPUs
In this article, authors discuss how to use the combination of Deep Java Learning (DJL), Apache Spark v3, and NVIDIA GPU computing to simplify deep learning pipelines while improving performance and reducing costs. They also show the performance comparison of this solution with GPU vs CPU hardware, using Amazon EMR and NVIDIA RAPIDS Accelerator.
-
/articles/azure-synapse-analytics-service/en/smallimage/evolution-of-azure-synapse-small-1622106447915.jpg)
Evolution of Azure Synapse: Apache Spark 3.0, GPU Acceleration, Delta Lake, Dataverse Support
At Microsoft Build 2021, Azure Synapse has announced significant improvements for its Apache Spark pool, its performance, and data querying and integration capabilities. This article outlines the improvements and provides the context.
-
/articles/storage-cloud-apache-bookkeeper/en/smallimage/logo-1619468572224.jpg)
Indestructible Storage in the Cloud with Apache Bookkeeper
At Salesforce, we required a storage system that could work with two kinds of streams, one stream for write-ahead logs and one for data. But we have competing requirements from both of the streams. Being the pioneers in cloud computing, we also required our storage system to be cloud-aware as the requirements of availability and durability are ever more increasing.
-
/articles/evolution-precomputation-technology-data-analytics/en/smallimage/the-evolutionof-precomputation-technology-small-1612883163091.jpg)
The Evolution of Precomputation Technology and its Role in Data Analytics
In this article, author Yang Li discusses the importance of precomputation techniques in databases, OLAP and data cubes, and some of the trends in using precomputation in big data analytics.
-
/articles/hive-performance-tuning-techniques/en/smallimage/data-table-s-1612367270351.jpeg)
Performance Tuning Techniques of Hive Big Data Table
In this article, author Sudhish Koloth discusses how to tackle performance problems when using Hive Big Data tables.
-
/articles/brain-not-computer/en/smallimage/the-biological-mind-s-1611697218890.jpg)
The Brain is Neither a Neural Network Nor a Computer: Book Review of The Biological Mind
Underlying much of artificial intelligence research is the idea that the essence of an individual resides in the brain. This is contrary to neuroscience which has discovered that a brain cannot work independently from the body and its environment. Understanding this enables us see what is reasonable to expect from artificial intelligence, as well as technology designed to improve human life.
-
/articles/overcoming-privacy-challenges-synthetic-data/en/smallimage/Overcoming-Data-Scarcity-Privacy-Challenges-with-Synthetic-Data-small-1608564047757.jpg)
Overcoming Data Scarcity and Privacy Challenges with Synthetic Data
In this article, the author discusses the importance of using synthetic data in data analytics projects, especially in financial institutions, to solve the problems of data scarcity and more importantly data privacy.
-
/articles/whats-the-next-step-for-data-management/en/smallimage/database-1605276684430.jpeg)
Beyond the Database, and beyond the Stream Processor: What's the Next Step for Data Management?
Databases have been around forever with the same shape: you make a request to your data and then you receive an answer. Now, stream processors came along with a different approach: data isn’t locked up, it is in motion. Understand how stream processors and databases relate and why there is an emerging new category of databases that focus on data that stays in place as well as data that moves.
-
/articles/privacy-shield-hyperscale-clouds/en/smallimage/privacy-shield-hyperscale-clouds-s-1602019588203.jpeg)
The End of the Privacy Shield Agreement Could Lead to Disaster for Hyperscale Cloud Providers
The recent ending of the Privacy Shield agreement by the European Court of Justice (ECJ) might impact cloud adoption. This article looks at the demise of this agreement, and possible solutions.
-
/articles/covid-social-media-machine-learning/en/smallimage/covid-social-media-machine-learning-s-1601562448288.jpeg)
COVID-19 and Mining Social Media - Enabling Machine Learning Workloads with Big Data
In this article, author Adi Pollock discusses how to enable machine learning workloads with big data to query and analyze COVID-19 tweets to understand social sentiment towards COVID-19.
-
/articles/cloud-to-cloudlets-data-processing/en/smallimage/cloud-to-cloudlets-data-processing-s-1601399656392.jpeg)
From Cloud to Cloudlets: a New Approach to Data Processing?
The growing popularity of small, distributed clouds, or “cloudlets” is an implicit recognition of the limitations of the “traditional” cloud model, and could signal a major shift in the way that data is collected, stored, and processed.
-
/articles/dataops-devops-scale-speed/en/smallimage/combining-dataops-devops-scale-at-speed-logo-small-1597065920955.jpg)
Combining DataOps and DevOps: Scale at Speed
DataOps is an extension of DevOps standards and processes into the data analytics world. It's about streamlining the processes involved in processing, analyzing and deriving value from big data.