InfoQ Homepage Database Content on InfoQ
-
AI, ML, and Data Engineering InfoQ Trends Report - September 2023
In this annual report, the InfoQ editors discuss the current state of AI, ML, and data engineering and what emerging trends you as a software engineer, architect, or data scientist should watch. We curate our discussions into a technology adoption curve with supporting commentary to help you understand how things are evolving.
-
Leveraging Eclipse JNoSQL 1.0.0: Quarkus Integration and Building a Pet-Friendly REST API
Eclipse JNoSQL 1.0.0 modernizes NoSQL integration with advanced features, standardized specs (Jakarta NoSQL & Jakarta Data), enhanced queries, schema migration, and Quarkus framework compatibility. It simplifies NoSQL use, boosts performance, scalability, and integrates seamlessly, empowering developers with tools to streamline data management in modern apps.
-
Designing the Jit Analytics Architecture for Scale and Reuse
As a SaaS provider, analytical data at Jit needs to be useful to both their customers and to internal stakeholders. AWS services including EventBridge, Kinesis Data Firehose, and Timestream handle data ingestion and UI platforms from Mixpanel and Segment provide data visualization.
-
In-Process Analytical Data Management with DuckDB
DuckDB is an open-source OLAP database for analytical data management that operates as an in-process database, avoiding data transfer overhead. Leveraging vectorized query processing and Morsel-Driven parallelism, the database optimizes performances and multi-core utilization for analytical data processing.
-
Debugging outside Your Comfort Zone: Diving beneath a Trusted Abstraction
This article takes a deep dive through a complex outage in the main database cluster of a payments company. We’ll focus on the aftermath of the incident - the process of understanding what went wrong, recreating the outage in a test cluster, and coming up with a way to stop it from happening again, and dive deep into the internals of Postgres, and learn about how it stores data on disk.
-
Minimising the Impact of Machine Learning on our Climate
This article introduces the field of green software engineering, showing the Green Software Foundation’s Software Carbon Intensity Specification, which is used to estimate the carbon footprint of software, and discusses ideas on how to make machine learning greener. It aims to give you the tools to take an active part in the climate solution.
-
Magic Pocket: Dropbox’s Exabyte-Scale Blob Storage System
A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud.
-
Banking on Thousands of Microservices
Lessons learned building a banking platform, starting from technological choices like using Cassandra and Kubernetes in the early days to maintain the speed of execution through platform engineering and developer experience. With some mistakes and incidents along the way.
-
Understanding and Applying Correspondence Analysis
Customer segments, personality profiles, social classes, and age generations are examples of effective references to larger groups of people sharing similar characteristics. Correspondence analysis (CA) is a multivariate analysis technique that projects categorical data into a numeric feature space which captures most of the variability in the data by fewer dimensions.
-
Data Protection Methods for Federal Organizations and beyond
The Federal Data Strategy describes a plan to “accelerate the use of data to deliver on mission, serve the public, and steward resources while protecting security, privacy, and confidentiality." This article covers what it is and how it can be applied to any organization.
-
InfoQ Software Trends Report: Major Trends in 2022 and What to Watch for in 2023
2022 was another year of significant technological innovations and trends in the software industry and communities. The InfoQ podcast co-hosts met last month to discuss the major trends from 2022, and what to watch for in 2023. This article is a summary of the 2022 software trends podcast.
-
Securing a Kafka Cluster in Kubernetes Using Strimzi
Deploying an Apache Kafka cluster to Kubernetes is easy if you use Strimzi, but that’s only the first step; you need to secure the communication between Kafka and the consumers and producers, provide RBAC to access topics, spread the secrets correctly to Kafka Connect components and all using a Kubernetes GitOps way.