InfoQ Homepage Big Data Content on InfoQ
-
/articles/data-protection-federal/en/smallimage/logo-1673874259012.jpg)
Data Protection Methods for Federal Organizations and beyond
The Federal Data Strategy describes a plan to “accelerate the use of data to deliver on mission, serve the public, and steward resources while protecting security, privacy, and confidentiality." This article covers what it is and how it can be applied to any organization.
-
/articles/anatomy-code-obfuscation/en/smallimage/logo-1667849003893.jpg)
Who Moved My Code? An Anatomy of Code Obfuscation
In this article, we introduce the topic of code obfuscation, with emphasis on string obfuscation. Obfuscation is an important practice to protect source code by making it unintelligible. Obfuscation is often mistaken with encryption, but they are different concepts. In the article we will present a number of techniques and approaches used to obfuscate data in a program.
-
/articles/transatlantic-data-privacy-2022-virtual-panel/en/smallimage/virtual-panel-the-new-US-EU-data-privacy-framework-small-image-1663768696087.jpg)
Virtual Panel: the New US-EU Data Privacy Framework
Recent rulings by several European courts have set important precedents for restricting personal data transmission from the EU to the US. As a consequence, the US and EU have started working on a new agreement. In this virtual panel, three knowledgeable experts discuss where the existing agreements fall short, and whether a new privacy agreement could improve the current situation.
-
/articles/cloud-native-apache-dolphinscheduler/en/smallimage/logo-1655898721435.jpg)
Embracing Cloud-Native for Apache DolphinScheduler with Kubernetes: a Case Study
This article shares how Apache DolphinScheduler was updated to use a more modern, cloud-native architecture. This includes moving to Kubernetes and integrating with Argo CD and Prometheus. This improves substantially the user experience of deploying, operating, and monitoring DolphinScheduler.
-
/articles/deep-learning-institutional-incremental-learning/en/smallimage/institutional-incremental-learning-small-image-v2-1640874831613.jpg)
Developing Deep Learning Systems Using Institutional Incremental Learning
Institutional incremental learning promises to achieve collaborative learning. This form of learning can address data sharing and security issues, without bringing in the complexities of federated learning. This article talks about practical approaches which help in building an object detection system.
-
/articles/deep-learning-apache-spark-nvidia-gpu/en/smallimage/spark-logo-1623166971929.jpg)
Accelerating Deep Learning on the JVM with Apache Spark and NVIDIA GPUs
In this article, authors discuss how to use the combination of Deep Java Learning (DJL), Apache Spark v3, and NVIDIA GPU computing to simplify deep learning pipelines while improving performance and reducing costs. They also show the performance comparison of this solution with GPU vs CPU hardware, using Amazon EMR and NVIDIA RAPIDS Accelerator.
-
/articles/azure-synapse-analytics-service/en/smallimage/evolution-of-azure-synapse-small-1622106447915.jpg)
Evolution of Azure Synapse: Apache Spark 3.0, GPU Acceleration, Delta Lake, Dataverse Support
At Microsoft Build 2021, Azure Synapse has announced significant improvements for its Apache Spark pool, its performance, and data querying and integration capabilities. This article outlines the improvements and provides the context.
-
/articles/storage-cloud-apache-bookkeeper/en/smallimage/logo-1619468572224.jpg)
Indestructible Storage in the Cloud with Apache Bookkeeper
At Salesforce, we required a storage system that could work with two kinds of streams, one stream for write-ahead logs and one for data. But we have competing requirements from both of the streams. Being the pioneers in cloud computing, we also required our storage system to be cloud-aware as the requirements of availability and durability are ever more increasing.
-
/articles/evolution-precomputation-technology-data-analytics/en/smallimage/the-evolutionof-precomputation-technology-small-1612883163091.jpg)
The Evolution of Precomputation Technology and its Role in Data Analytics
In this article, author Yang Li discusses the importance of precomputation techniques in databases, OLAP and data cubes, and some of the trends in using precomputation in big data analytics.
-
/articles/hive-performance-tuning-techniques/en/smallimage/data-table-s-1612367270351.jpeg)
Performance Tuning Techniques of Hive Big Data Table
In this article, author Sudhish Koloth discusses how to tackle performance problems when using Hive Big Data tables.
-
/articles/brain-not-computer/en/smallimage/the-biological-mind-s-1611697218890.jpg)
The Brain is Neither a Neural Network Nor a Computer: Book Review of The Biological Mind
Underlying much of artificial intelligence research is the idea that the essence of an individual resides in the brain. This is contrary to neuroscience which has discovered that a brain cannot work independently from the body and its environment. Understanding this enables us see what is reasonable to expect from artificial intelligence, as well as technology designed to improve human life.
-
/articles/overcoming-privacy-challenges-synthetic-data/en/smallimage/Overcoming-Data-Scarcity-Privacy-Challenges-with-Synthetic-Data-small-1608564047757.jpg)
Overcoming Data Scarcity and Privacy Challenges with Synthetic Data
In this article, the author discusses the importance of using synthetic data in data analytics projects, especially in financial institutions, to solve the problems of data scarcity and more importantly data privacy.