InfoQ Homepage Natural Language Processing Content on InfoQ
-
Google Announces AI-Generated Summaries for Google Docs
Google has announced a new feature for their Docs app that will automatically generate a summary of the document content. The summarization is powered by a natural language processing (NLP) AI model based on the Transformer architecture.
-
EleutherAI Open-Sources 20 Billion Parameter AI Language Model GPT-NeoX-20B
Researchers from EleutherAI have open-sourced GPT-NeoX-20B, a 20-billion parameter natural language processing (NLP) AI model similar to GPT-3. The model was trained on 825GB of publicly available text data and has performance comparable to similarly-sized GPT-3 models.
-
Deep Learning Toolkit Intel OpenVINO Extends API, Improves Performance, and More
The latest release of Intel OpenVINO offers a cleaner API, expands support for natural language processing, and improves performance and portability thanks to its new AUTO plugin. InfoQ has spoken with senior director AI Intel OpenVINO Matthew Formica to learn more.
-
AlphaCode: Competitive Code Synthesis with Deep Learning
AlphaCode study brings promising results for goal-oriented code synthesis using deep sequence-to-sequence models. It extends the previous networks and releases a new dataset named CodeContests to contribute to future research benchmarks.
-
Tel-Aviv University Releases Long-Text NLP Benchmark SCROLLS
Researchers with Tel-Aviv University, Meta AI, IBM Research, and Allen Institute for AI have released Standardized CompaRison Over Long Language Sequences (SCROLLS), a set of natural language processing (NLP) benchmark tasks operating on long text sequences drawn from many domains. Experiments on baseline NLP models show that current models have significant room for improvement.
-
OpenAI Introduces InstructGPT Language Model to Follow Human Instructions
OpenAI overhauled the GPT-3 language model and introduced a new default tool called InstructGPT to address complaints about toxic language and misinformation.
-
OpenAI Announces Question-Answering AI WebGPT
OpenAI has developed WebGPT, an AI model for long-form question-answering based on GPT-3. WebGPT can use web search queries to collect supporting references for its response, and on Reddit questions its answers were preferred by human judges over the highest-voted answer 69% of the time.
-
AI Listens by Seeing as Well
Meta AI released a self-supervised speech recognition model that also uses video and achieves 75% better accuracy for some amount of data than current state-of-the-art models. This new model, Audio-Visual Hidden BERT (AV-HuBERT), uses audiovisual features for improving models based only on hearing speech. Visual features used are based on lip-reading, similar to what humans do.
-
Facebook Open-Sources Two Billion Parameter Multilingual Speech Recognition Model XLS-R
Facebook AI Research (FAIR) open-sourced XLS-R, a cross-lingual speech recognition (SR) AI model. XSLR is trained on 436K hours of speech audio from 128 languages, an order of magnitude more than the largest previous models, and outperforms the current state-of-the-art on several downstream SR and translation tasks.
-
Google Trains 280 Billion Parameter AI Language Model Gopher
Google subsidiary DeepMind announced Gopher, a 280-billion-parameter AI natural language processing (NLP) model. Based on the Transformer architecture and trained on a 10.5TB corpus called MassiveText, Gopher outperformed the current state-of-the-art on 100 of 124 evaluation tasks.
-
Facebook Develops New AI Model That Can Anticipate Future Actions
Facebook unveiled its latest machine-learning process called Anticipative Video Transformer (AVT), which is able to predict future actions by using visual interpretation. AVT works as an end-to-end attention-based model for action anticipation in videos.
-
BigScience Research Workshop Releases AI Language Model T0
BigScience Research Workshop released T0, a series of natural language processing (NLP) AI models specifically trained for researching zero-shot multitask learning. T0 can often outperform models 6x larger on the BIG-bench benchmark, and can outperform the 16x larger GPT-3 on several other NLP benchmarks.
-
Baidu Announces 11 Billion Parameter Chatbot AI PLATO-XL
Baidu recently announced PLATO-XL, an AI model for dialog generation, which was trained on over a billion samples collected from social media conversations in both English and Chinese. PLATO-XL achieves state-of-the-art performance on several conversational benchmarks, outperforming currently available commercial chatbots.
-
Baidu's ERNIE 3.0 AI Model Exceeds Human Performance on Language Understanding Benchmark
A research team from Baidu published a paper on the 3.0 version of Enhanced Language RepresentatioN with Informative Entities (ERNIE), a natural language processing (NLP) deep-learning model. The model contains 10B parameters and achieved a new state-of-the-art score on the SuperGLUE benchmark, outperforming the human baseline score.
-
Google Announces 800M Parameter Vision-Language AI Model ALIGN
Google Research announced the development of A Large-scale ImaGe and Noisy-Text Embedding (ALIGN), an 800M-parameter pre-trained deep-learning model trained on a noisy dataset of 1.8B image-text pairs. The model can be used on several downstream tasks and achieves state-of-the-art accuracy on several image-text retrieval benchmarks.