InfoQ Homepage Natural Language Processing Content on InfoQ
-
Transformers Can Mock Part of Human Brain
In recent years, neuroscientists have tried many types of neural networks to model the firing of neurons in the human brain. In a recent project, two researchers Whittington and Behrens found that the hippocampus, a structure of the brain critical to memory, works as a particular kind of artificial neural network called transformer.
-
Microsoft Trains Two Billion Parameter Vision-Language AI Model BEiT-3
Researchers from Microsoft's Natural Language Computing (NLC) group announced the latest version of Bidirectional Encoder representation from Image Transformers: BEiT-3, a 1.9B parameter vision-language AI model. BEiT-3 models images as another language and achieves state-of-the-art performance on a wide range of downstream tasks.
-
Google Open-Sources Natural Language Robot Control Method SayCan
Researchers from Google's Robotics team have open-sourced SayCan, a robot control method that uses a large language model (LLM) to plan a sequence of robotic actions to achieve a user-specified goal. In experiments, SayCan generated the correct action sequence 84% of the time.
-
Amazon's AlexaTM 20B Model Outperforms GPT-3 on NLP Benchmarks
Researchers at Amazon Alexa AI have announced Alexa Teacher Models (AlexaTM 20B), a 20-billion-parameter sequence-to-sequence (seq2seq) language model that exhibits state-of-the-art performance on 1-shot and few-shot NLP tasks. AlexaTM 20B outperforms GPT-3 on SuperGLUE and SQuADv2 benchmarks while having fewer than 1/8 the number of parameters.
-
BigScience Releases 176B Parameter AI Language Model BLOOM
The BigScience research workshop released BigScience Large Open-science Open-access Multilingual Language Model (BLOOM), an autoregressive language model based on the GPT-3 architecture. BLOOM is trained on data from 46 natural languages and 13 programming languages and is the largest publicly available open multilingual model.
-
Google AI Developed a Language Model to Solve Quantitative Reasoning Problems
Google AI developed a deep learning language model called Minerva which could solve mathematical quantitative problems. Google AI researchers achieved a state-of-the-art deep learning model by training on a large dataset that contains quantitative reasoning with symbolic expressions. The final model, Minerva, could solve quantitative mathematical problems on STEM reasoning tasks.
-
Stanford University Open-Sources Controllable Generative Language AI Diffusion-LM
Researchers at Stanford University have open-sourced Diffusion-LM, a non-autoregressive generative language model that allows for fine-grained control of the model's output text. When evaluated on controlled text generation tasks, Diffusion-LM outperforms existing methods.
-
DeepMind Trains 80 Billion Parameter AI Vision-Language Model Flamingo
DeepMind recently trained Flamingo, an 80B parameter vision-language model (VLM) AI. Flamingo combines separately pre-trained vision and language models and outperforms all other few-shot learning models on 16 vision-language benchmarks. Flamingo can also chat with users, answering questions about input images and videos.
-
Meta Open-Sources 175 Billion Parameter AI Language Model OPT
Meta AI Research released Open Pre-trained Transformer (OPT-175B), a 175B parameter AI language model. The model was trained on a dataset containing 180B tokens and exhibits performance comparable with GPT-3, while only requiring 1/7th GPT-3's training carbon footprint.
-
Amazon Releases 51-Language AI Training Dataset MASSIVE
Amazon Alexa AI's Natural Language Understanding group released Multilingual Amazon SLURP (SLU resource package) for Slot Filling, Intent Classification, and Virtual-Assistant Evaluation (MASSIVE), a dataset for training natural language understanding (NLU) AI models that contains one million annotated samples from 51 languages. The release also includes code and tools for using the data.
-
Google Trains 540 Billion Parameter AI Language Model PaLM
Google Research recently announced the Pathways Language Model (PaLM), a 540-billion-parameter AI natural language processing (NLP) model that surpasses average human performance on the BIG-bench benchmark. PaLM outperforms other state-of-the-art systems on many evaluation tasks, and shows strong results on tasks such as logical inference and joke explanation.
-
Google Announces AI-Generated Summaries for Google Docs
Google has announced a new feature for their Docs app that will automatically generate a summary of the document content. The summarization is powered by a natural language processing (NLP) AI model based on the Transformer architecture.
-
EleutherAI Open-Sources 20 Billion Parameter AI Language Model GPT-NeoX-20B
Researchers from EleutherAI have open-sourced GPT-NeoX-20B, a 20-billion parameter natural language processing (NLP) AI model similar to GPT-3. The model was trained on 825GB of publicly available text data and has performance comparable to similarly-sized GPT-3 models.
-
Deep Learning Toolkit Intel OpenVINO Extends API, Improves Performance, and More
The latest release of Intel OpenVINO offers a cleaner API, expands support for natural language processing, and improves performance and portability thanks to its new AUTO plugin. InfoQ has spoken with senior director AI Intel OpenVINO Matthew Formica to learn more.
-
AlphaCode: Competitive Code Synthesis with Deep Learning
AlphaCode study brings promising results for goal-oriented code synthesis using deep sequence-to-sequence models. It extends the previous networks and releases a new dataset named CodeContests to contribute to future research benchmarks.