InfoQ Homepage Big Data Content on InfoQ
-
Netflix Announces Genie 3
Netflix announced major revisions and functionality in their Big Data distributed workflow management tool, Genie 3. In its newest version, Genie 3 supports scalable, config-driven data processing executables and task pipelines.
-
Scalable Chatbot Architecture with eBay ShopBot Shopping Assistant
Robert Enyedi, software engineer at eBay spoke at QCon New York 2017 Conference about ShopBot personal shopping assistant application. ShopBot, launched in late 2016 based on Facebook Messenger bot, leverages AI components and the eBay user data to provide shopping options in a conversational style.
-
Enhancing Google Maps with Deep Learning and Street View
Google's Ground Truth team recently announced a new Deep Learning model for the automatic extraction of information from geo-located image files to improve Google Maps. This neural network model achieved a higher accuracy in processing the challenging French Street Name Signs (FSNS) dataset.
-
Facebook Publishes New Neural Machine Translation Algorithm
Facebook’s Artificial Intelligence Research team published research results using a new approach for neural machine translation (NMT). Their algorithm scores higher than any other system on three established machine translation tasks.
-
Developing Virtual Assistant Apps with Amazon Lex and Polly Deep Learning Technologies
Greg Bulmash from Amazon spoke at the OSCON 2017 Conference last week about developing your own virtual assistant applications using Amazon's Lex and Polly technologies.
-
Apache Metron Graduates to Top-Level Project
Hortonworks and Apache announce graduation of Metron, a realtime big data security platform to top-level project at the ASF.
-
Confluent Cloud, Apache Kafka as a Service in AWS
Apache Kafka is a distributed, fault-tolerant pub sub messaging soltuion, originally developed by LinkedIn and open sourced. Confluent was formed by former LinkedIn engineers in the Kafka development group and today announced Confluent Cloud, a fully hosted and managed Apache Kafka as a Service in AWS. We also take a look at Confluent's second annual Streaming Data report and its findings.
-
Data Preparation Pipelines: Strategy, Options and Tools
Data preparation is an important aspect of data processing and analytics use cases. Business analysts and data scientists spend about 80% of their time gathering and preparing the data rather than analyzing it or developing machine learning models. Kelly Stirman spoke last week at Enterprise Data World 2017 Conference about the data preparation best practices.
-
Google Announces Cloud Machine Learning API Updates
Google recently announced the Cloud Machine Learning API updates at the Google Cloud Next Conference. This includes a set of APIs in the areas of vision, video intelligence, speech, natural language, translation and job search.
-
Using Deep Learning Technologies IBM Reaches a New Milestone in Speech Recognition
The research team at IBM recently announced they've reached a new industry record at 5.5%, using the SWITCHBOARD linguistic corpus. This brings us closer to what's considered to be the human error rate, 5.1%. They used deep learning technologies and acoustic models to accomplish this milestone.
-
Netflix Demonstrates Big Data Analytics Infrastructure
At QCon San Francisco, engineers at Netflix discussed their big data strategy and analytics infrastructure. This included a summary of the scale of their data, their S3 data warehouse, and Genie, their big data federated orchestration system.
-
Apache Ranger Graduates to Top-Level Project
Apache Ranger, a security management framework for Apache Hadoop ecosystem, graduated to top level. Ranger is used as a centralized component to define and administer security policies that are enforced across supported Hadoop components such as Apache HBase, Hadoop (HDFS and YARN), Apache Hive, Apache Kafka, Apache Solr, among others.
-
Beam Graduates to Top-Level Apache Project
Beam exits incubation period and graduates to top-level Apache project, Google support and contribution to open source integration for various data processing backends and more.
-
Deep Learning at Gilt
Deep Learning is a rapidly evolving subfield of Machine Learning originating from Neural Networks. Recent algorithmic advances and utilization of GPU parallelization have resulted in Deep Learning based algorithms mastering the game of Go as well as several practical applications. The fashion industry is one of the target sectors for Deep Learning. Gilt is using Deep Learning for real world apps
-
Microsoft AirSim, a Simulator for Drones and Robots
Microsoft has developed and open sourced AirSim, a tool that can be used to simulate the flight of drones around the world. The simulator is built on the Unreal Engine and Microsoft will soon add support for robots and other types of vehicles.