InfoQ Homepage Natural Language Processing Content on InfoQ
-
Google AI Launches NLU-Powered Tool to Help Explore COVID-19 Literature
Google AI launched COVID-19 Research Explorer, which provides a semantic search interface on top of the COVID-19 Open Research Dataset to help scientists and researchers efficiently analyze all of the dataset’s journal articles and preprints.
-
Blender, Facebook State-of-the-Art Human-Like Chatbot, Now Open Source
Blender is an open-domain chatbot developed at Facebook AI Research (FAIR), Facebook’s AI and machine learning division. According to FAIR, it is the first chatbot that has learned to blend several conversation skills, including the ability to show empathy and discuss nearly any topic, beating Google's chatbot in tests with human evaluators.
-
OpenAI Approximates Scaling Laws for Neural Language Models
Artificial intelligence company OpenAI studies empirical scaling laws for language models using cross entropy loss to determine the optimal allocation of a fixed compute budget.
-
Microsoft and Google Release New Benchmarks for Cross-Language AI Tasks
Research teams at Microsoft Research and Google AI have announced new benchmarks for cross-language natural-language understanding (NLU) tasks for AI systems, including named-entity recognition and question answering. Google's XTREME covers 40 languages and includes nine tasks, while Microsoft's XGLUE covers 27 languages and eleven tasks.
-
ExBERT, a Tool for Exploring Learned Representations in NLP Models
MIT-IBM AI Labs and Harvard NLP Group have released a live demo of their interactive visualization tool for exploring learned representations in Transformers models called exBERT, along with a pre-publication and the source-code.
-
Stanford NLP Group Releases Stanza: a Python NLP Toolkit
The Stanford NLP Group recently released Stanza, a new python natural language processing toolkit. Stanza features both a language-agnostic fully neural pipeline for text analysis (supporting 66 human languages), and a Python interface to the Java Stanford CoreNLP software.
-
Google Open-Sources Reformer Efficient Deep-Learning Model
Researchers from Google AI recently open-sourced the Reformer, a more efficient version of the Transformer deep-learning model. Using a hashing trick for attention calculation and reversible residual layers, the Reformer can handle text sequences up to 1 million words while consuming only 16GB of memory on a single GPU accelerator.
-
Microsoft Open-Sources ONNX Acceleration for BERT AI Model
Microsoft's Azure Machine Learning team recently open-sourced their contribution to the ONNX Runtime library for improving the performance of the natural language processing (NLP) model BERT. With the optimizations, the model's inference latency on the SQUAD benchmark sped up 17x.
-
Google Open-Sources ALBERT Natural Language Model
Google AI has open-source A Lite Bert (ALBERT), a deep-learning natural language processing (NLP) model, which uses 89% fewer parameters than the state-of-the-art BERT model, with little loss of accuracy. The model can also be scaled-up to achieve new state-of-the-art performance on NLP benchmarks.
-
Uber Open-Sources Plug-and-Play Language Model for Controlling AI-Generated Text
Uber AI open-sourced the plug-and-play language model (PPLM) which can control the topic and sentiment of AI-generated text. The model's output is evaluated by human judges as achieving 36% better topic accuracy compared to the baseline GPT-2 model.
-
Google Cloud Team Releases AutoML Natural Language
The Google Cloud team recently announced the generally available (GA) release of AutoML Natural Language framework. AutoML Natural Language supports features for data processing and common machine learning tasks like classification, sentiment analysis, and entity extraction.
-
Microsoft Releases DialogGPT AI Conversation Model
Microsoft Research's Natural Language Processing Group released dialogue generative pre-trained transformer (DialoGPT), a pre-trained deep-learning natural language processing (NLP) model for automatic conversation response generation. The model was trained on over 147M dialogues and achieves state-of-the-art results on several benchmarks.
-
Google Applies NLP Algorithm BERT to Search
BERT, Google's latest NLP algorithm, will power Google search and make it better at understanding user queries in a way more similar to how humans would understand them, writes Pandu Nayak, Google fellow and vice president for Search, with one in 10 queries providing a different set of results.
-
Microsoft and University of Maryland Researchers Announce FreeLB Adversarial Training System
Researchers from Microsoft and the University of Maryland (UMD) announced Free Large-Batch (FreeLB), a new adversarial training technique for deep-learning natural-language processing (NLP) systems that improves accuracy, increasing RoBERTa's scores on the General Language Understanding Evaluation (GLUE) benchmark and achieving the highest score on AI2 Reasoning Challenge (ARC) benchmark.
-
Facebook Releases AI Code Search Datasets
Facebook AI released a dataset containing coding questions paired with code-snippet answers, intended for evaluating AI-based natural-language code search systems. The release also includes benchmark results for several of Facebook's own code-search models and a training corpus of over 4 million Java methods parsed from over 24,000 GitHub repositories.