InfoQ Homepage Apache Content on InfoQ

Articles

RSS Feed

Newer Older

AI, ML & Data Engineering

Create Your Distributed Database on Kubernetes with Existing Monolithic Databases

The next challenge for databases is to run them on Kubernetes to become cloud neutral. However, they are more difficult to manage than the application layer, since Kubernetes is designed for stateless applications. Apache ShardingSphere is the ecosystem to transform any database into a distributed database system and enhance it with sharding, elastic scaling, encryption features, and more.

Trista Pan
on Nov 16, 2022
Architecture & Design

Creating a Secure Distributed Database Cluster Leveraging Your Existing Database Management System

The emergence of Big Data and data lakes doesn't necessarily mean the disappearance of the trusted relational database. The two can coexist, relational databases just need to adjust. For the transition we propose Database Plus, a new technology & concept applicable to any database, that answers these challenges and eliminates switching costs and vendor lock-in.

Trista Pan
on Aug 17, 2022
Java

Introduction to Apache Beam Using Java

Apache Beam is a stream processor, helping developers migrate work between different processes to offload work onto runners that leverage external resources.

Fabio Hiroki
on Jun 06, 2022
AI, ML & Data Engineering

The Next Evolution of the Database Sharding Architecture

In this article, author Juan Pan discusses the data sharding architecture patterns in a distributed database system. She explains how Apache ShardingSphere project solves the data sharding challenges. Also discussed are two practical examples of how to create a distributed database and an encrypted table with DistSQL.

Trista Pan
on Jan 12, 2022
Architecture & Design

Building Latency Sensitive User Facing Analytics via Apache Pinot

At QCon, a virtual conference for senior software engineers and architects covering the trends, Chinmay Soman talked about how you can use Apache Pinot as part of your data pipelines for building rich, external, or site-facing analytics.

Chinmay Soman
on Jun 28, 2021
Cloud

Indestructible Storage in the Cloud with Apache Bookkeeper

At Salesforce, we required a storage system that could work with two kinds of streams, one stream for write-ahead logs and one for data. But we have competing requirements from both of the streams. Being the pioneers in cloud computing, we also required our storage system to be cloud-aware as the requirements of availability and durability are ever more increasing.

Anup Ghatage
on Apr 28, 2021
AI, ML & Data Engineering

Apache Beam Interview with Frances Perry

InfoQ Interviews Apache Beam's Frances Perry about the impetus for using Beam and the future of the top-level open source project and covers the thoughts behind the programming model as well as some of the touch-points in integration with other data engineering tools like Apache Spark and Flink.

Dylan Raithel
on Jun 20, 2017
AI, ML & Data Engineering

From Alibaba to Apache: RocketMQ’s Past, Present, and Future

Feng Jia and Wang Xiaorui share the core distributed systems principals behind RocketMQ, Alibaba's distributed messaging and data streaming platform now open sourced through the Apache Foundation.

Wang Xiaorui Feng Jia
on Apr 21, 2017
AI, ML & Data Engineering

Article Series: Getting a Handle on Data Science as a Software Developer

Software developers and managers are realizing that they need data science among their skills, to be able to tackle pressing problems. In this series, field experts provide guidance to help us navigate among the available data analysis options. They explore ways of understanding where data science is needed and where it’s not, and how to turn it into an asset.

Francine Bennett
on Dec 05, 2016
AI, ML & Data Engineering

Case Study: Selecting Big Data and Data Science Technologies at a large Financial Organisation

Adopting Big Data and Data Science technologies into an organisation is a transformative project similar to an agile transformation and with many similar challenges. In this article, the author describes such a project for a FTSE100 financial services company.

Christian Prokopp
on Nov 10, 2016
Development

Q&A: Relevant Search with Elasticsearch and Solr

In their book "Relevant Search", Doug Turnbull and John Berryman focus on the challenge of providing search results by balancing the needs and intents of the user. Using Elasticsearch and Solr, relevance engineers can constantly tune the needs of the business vs. the needs of the user.

David Iffland
on Sep 26, 2016
Web Development

Bridging Microsoft Word and the Browser

HTML editors work fine for general formatting, but they don’t have all the capabilities that some businesses require. Creating graphics, diagrams, tracking changes and inserting comments are useful and come out of the box in Microsoft Word In this article, Prasadu Babu Dandu shows how to serve up Word documents as HTML.

Prasadu Babu Dandu
on Oct 26, 2015