InfoQ Homepage Big Data Infrastructure Content on InfoQ
Interviews
RSS Feed-
Chris Mattmann on Big Data Infrastructure for Scientific Data Processing
Chris Mattmann explains the type and magnitude of data produced in scientific projects like the Square Kilometer Array Telescope, the tools to use for scientific data processing and much more.
-
Machine Learning Netflix Style with Xavier Amatriain
Xavier Amatriain discusses how Netflix uses specialized roles, including that of the Data Scientist and Machine Learning Engineer, to deliver valuable data at the right time to Netflix' customer base through a mixture of offline, online, and nearline data processes. Xavier also discusses what it takes to become a Machine Learning Engineer and how to gain real experience in the field.
-
Optimizing for Big Data at Facebook
Hive co-creator Ashish Thusoo describes the Big Data challenges Facebook faced and presents solutions in 2 areas: Reduction in the data footprint and CPU utilization. Generating 300 to 400 terabytes per day, they store RC files as blocks, but store as columns within a block to get better compression. He also talks about the current Big Data ecosystem and trends for companies going forward.
