InfoQ Homepage Neural Networks Content on InfoQ
-
Stanford University Publishes AI Index 2022 Annual Report
Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) has published its 2022 AI Index annual report. The report identifies top trends in AI, including advances in technical achievements, a sharp increase in private investment, and increasing attention on ethical issues.
-
EleutherAI Open-Sources 20 Billion Parameter AI Language Model GPT-NeoX-20B
Researchers from EleutherAI have open-sourced GPT-NeoX-20B, a 20-billion parameter natural language processing (NLP) AI model similar to GPT-3. The model was trained on 825GB of publicly available text data and has performance comparable to similarly-sized GPT-3 models.
-
University of Washington Open-Sources AI Fine-Tuning Algorithm WISE-FT
A team of researchers from University of Washington (UW), Google Brain, and Columbia University have open-sourced weight-space ensembles for fine-tuning (WiSE-FT), an algorithm for fine-tuning AI models that improves robustness under distribution shift. Experiments on several computer vision (CV) benchmarks show that WISE-FT improves accuracy up to 6 percentage points.
-
University Researchers Investigate Machine Learning Compute Trends
A team of researchers from University of Aberdeen, MIT, and several other institutions have released a dataset of historical compute demands for machine learning (ML) models. The dataset contains the compute required for training 123 important models, and an analysis shows that since the year 2010 the trend has significantly increased.
-
Waymo Releases Block-NeRF 3D View Synthesis Deep-Learning Model
Waymo released a ground-breaking deep-learning model called Block-NeRF for large-scale 3D world-view synthesis reconstructed from images collected by its self-driving cars. NeRF has the ability to encode surface and volume representation in neural networks.
-
Meta Open-Sources Multi-Modal AI Algorithm Data2vec
Meta AI recently open-sourced data2vec, a unified framework for self-supervised deep learning on images, text, and speech audio data. When evaluated on common benchmarks, models trained using data2vec perform as well as or better than state-of-the-art models trained with modality-specific objectives.
-
How GitHub Uses Machine Learning to Extend Vulnerability Code Scanning
Applying machine learning techniques to its rule-based security code scanning capabilities, GitHub hopes to be able to extend them to less common vulnerability patterns by automatically inferring new rules from the existing ones.
-
DeepMind Open-Sources Quantum Chemistry AI Model DM21
Researchers at Google subsidiary DeepMind have open-sourced DM21, a neural network model for mapping electron density to chemical interaction energy, a key component of quantum mechanical simulation. DM21 outperforms traditional models on several benchmarks and is available as an extension to the PySCF simulation framework.
-
Alibaba Open-Sources AutoML Algorithm KNAS
Researchers from Alibaba Group and Peking University have open-sourced Kernel Neural Architecture Search (KNAS), an efficient automated machine learning (AutoML) algorithm that can evaluate proposed architectures without training. KNAS uses a gradient kernel as a proxy for model quality, and uses an order of magnitude less compute power than baseline methods.
-
LambdaML: Pros and Cons of Serverless for Deep Network Training
A new study entitled "Towards Demystifying Serverless Machine Learning Training" aims to provide an experimental analysis of training deep networks by leveraging serverless platforms. FaaS for training has challenges due to its distributed nature and aggregation step in the learning algorithms. Results indicate FaaS can be a faster (for lightweight models) but not cheaper alternative than IaaS.
-
Meta AI’s Convolution Networks Upgrade Improves Image Classification
Meta AI released a new generation of improved Convolution Networks, achieving state-of-the-art performance of 87.8% accuracy on Image-Net top-1 dataset and outperforming Swin Transformers on COCO dataset where object detection performance is evaluated. The new design and training approach is inspired by the Swin Transformers model.
-
Evaluating Continual Deep Learning: a New Benchmark for Image Classification
Continual learning aims to preserve knowledge across deep network training iterations. A new dataset entitled "The CLEAR Benchmark: Continual LEArning on Real-World Imagery" has recently been published. The goal of the study is to establish a consistent image classification benchmark with the natural time evolution of objects for a more realistic comparison of continual learning models.
-
OpenAI Announces Question-Answering AI WebGPT
OpenAI has developed WebGPT, an AI model for long-form question-answering based on GPT-3. WebGPT can use web search queries to collect supporting references for its response, and on Reddit questions its answers were preferred by human judges over the highest-voted answer 69% of the time.
-
DeepMind Releases Weather Forecasting AI Deep Generative Models of Rainfall
DeepMind open-sourced a dataset and trained model snapshot for Deep Generative Models of Rainfall (DGMR), an AI system for short-term precipitation forecasts. In evaluations conducted by 58 expert meteorologists comparing it to other existing methods, DGMR was ranked first in accuracy and usefulness in 89% of test cases.
-
MIT Researchers Investigate Deep Learning's Computational Burden
A team of researchers from MIT, Yonsei University, and University of Brasilia have launched a new website, Computer Progress, which analyzes the computational burden from over 1,000 deep learning research papers. Data from the site show that computational burden is growing faster than the expected rate, suggesting that algorithms still have room for improvement.