InfoQ Homepage Partitioning Content on InfoQ
News
RSS Feed-
Pinterest Uses Event Time Watermarks to Finalize Petabyte Scale Data Partitions
Pinterest has introduced event time-based partition finalization for its database ingestion framework. The mechanism uses Apache Flink to track event time and Apache Iceberg metadata to publish a non-regressing watermark, helping downstream jobs determine when continuously ingested CDC data is sufficiently complete for processing.
-
Netflix Cuts Cassandra Read Latency from Seconds to Milliseconds with Dynamic Partition Splitting
Netflix engineers introduced dynamic partition splitting for Cassandra to address wide partitions in time series workloads. The metadata-driven approach detects oversized partitions, splits them smaller units, and routes reads across child partitions. Netflix reported lower read latency from seconds to milliseconds, reduced timeouts, and improved cluster stability while maintaining transparency.
-
Google Cloud Introduces Geo-Partitioning for Spanner: Reduced Latency and Cost Optimization
Google Cloud has announced adding geo-partitioning to Spanner, its fully-managed, globally distributed database. According to the company, this new feature aims to improve performance and user experience for geographically dispersed applications and users while optimizing operational costs.
-
Alternatives to Eventual Consistency
Causal Consistency models offer an alternative Eventual Consistency for distributed systems; both models should be weighed against your system's requirements and risk tolerance.