InfoQ Homepage Natural Language Processing Content on InfoQ
-
OpenAI Unveils a Powerful, Cost-Effective, and User-Friendly Embedding Model
OpenAI is introducing text-embedding-ada-002, a cutting-edge embedding model that combines the capabilities of five previous models for text search, text similarity, and code search. This new model outperforms the previous most capable model, Davinci, on most tasks, while being significantly more cost-effective at 99.8% lower pricing.
-
OpenAI Releases Conversational AI Model ChatGPT
OpenAI released ChatGPT, a conversational AI model based on their GPT-3.5 language model (LM). ChatGPT is fine-tuned using Reinforcement Learning from Human Feedback (RLHF) and includes a moderation filter to block inappropriate interactions.
-
Google's Code-as-Policies Lets Robots Write Their Own Code
Researchers from Google's Robotics team have open-sourced Code-as-Policies (CaP), a robot control method that uses a large language model (LLM) to generate robot-control code that achieves a user-specified goal. CaP uses a hierarchical prompting technique for code generation that outperforms previous methods on the HumanEval code-generation benchmark.
-
Galactica: Large Language Model for Scientific Knowledge
Meta AI and Papers with Code recently released Galactica, a 120-billion-parameter scientific-language model which can search and summarize academic literature, solve math problems, and write scientific code. Galactica’s architecture is based on a transformer, an attention mechanism which draws global dependencies between input and output.
-
Salesforce Open-Sources Language-Vision AI Toolkit LAVIS
Salesforce Research recently open-sourced LAnguage-VISion (LAVIS), a unified library for deep-learning language-vision research. LAVIS supports more than 10 language-vision tasks on 20 public datasets and includes pre-trained model weights for over 30 fine-tuned models.
-
University Researchers Publish Results of NLP Community Metasurvey
Researchers from New York University, University of Washington, and Johns Hopkins University have published the results of the NLP Community Metasurvey, which compiles the opinions of 480 active NLP researchers about several issues in the natural language processing AI field. The survey also includes meta-questions about the perceived opinions of other researchers.
-
Transformers Can Mock Part of Human Brain
In recent years, neuroscientists have tried many types of neural networks to model the firing of neurons in the human brain. In a recent project, two researchers Whittington and Behrens found that the hippocampus, a structure of the brain critical to memory, works as a particular kind of artificial neural network called transformer.
-
Microsoft Trains Two Billion Parameter Vision-Language AI Model BEiT-3
Researchers from Microsoft's Natural Language Computing (NLC) group announced the latest version of Bidirectional Encoder representation from Image Transformers: BEiT-3, a 1.9B parameter vision-language AI model. BEiT-3 models images as another language and achieves state-of-the-art performance on a wide range of downstream tasks.
-
Google Open-Sources Natural Language Robot Control Method SayCan
Researchers from Google's Robotics team have open-sourced SayCan, a robot control method that uses a large language model (LLM) to plan a sequence of robotic actions to achieve a user-specified goal. In experiments, SayCan generated the correct action sequence 84% of the time.
-
Amazon's AlexaTM 20B Model Outperforms GPT-3 on NLP Benchmarks
Researchers at Amazon Alexa AI have announced Alexa Teacher Models (AlexaTM 20B), a 20-billion-parameter sequence-to-sequence (seq2seq) language model that exhibits state-of-the-art performance on 1-shot and few-shot NLP tasks. AlexaTM 20B outperforms GPT-3 on SuperGLUE and SQuADv2 benchmarks while having fewer than 1/8 the number of parameters.
-
BigScience Releases 176B Parameter AI Language Model BLOOM
The BigScience research workshop released BigScience Large Open-science Open-access Multilingual Language Model (BLOOM), an autoregressive language model based on the GPT-3 architecture. BLOOM is trained on data from 46 natural languages and 13 programming languages and is the largest publicly available open multilingual model.
-
Google AI Developed a Language Model to Solve Quantitative Reasoning Problems
Google AI developed a deep learning language model called Minerva which could solve mathematical quantitative problems. Google AI researchers achieved a state-of-the-art deep learning model by training on a large dataset that contains quantitative reasoning with symbolic expressions. The final model, Minerva, could solve quantitative mathematical problems on STEM reasoning tasks.
-
Stanford University Open-Sources Controllable Generative Language AI Diffusion-LM
Researchers at Stanford University have open-sourced Diffusion-LM, a non-autoregressive generative language model that allows for fine-grained control of the model's output text. When evaluated on controlled text generation tasks, Diffusion-LM outperforms existing methods.
-
DeepMind Trains 80 Billion Parameter AI Vision-Language Model Flamingo
DeepMind recently trained Flamingo, an 80B parameter vision-language model (VLM) AI. Flamingo combines separately pre-trained vision and language models and outperforms all other few-shot learning models on 16 vision-language benchmarks. Flamingo can also chat with users, answering questions about input images and videos.
-
Meta Open-Sources 175 Billion Parameter AI Language Model OPT
Meta AI Research released Open Pre-trained Transformer (OPT-175B), a 175B parameter AI language model. The model was trained on a dataset containing 180B tokens and exhibits performance comparable with GPT-3, while only requiring 1/7th GPT-3's training carbon footprint.