Meta AI has developed AITemplate (AIT), a unified open-source system with separate acceleration back-ends for both AMD and NVIDIA GPU hardware technology. AITemplate (AIT) is a two-part Python framework for AI models that transforms them into much faster C++ code. It has a front-end that optimizes models through graph transformations and optimizations.
When it comes to providing computational capacity for the use of AI models, GPUs are very important, especially for large-scale pretrained models. The selection of high-performance GPU inference solutions by AI practitioners is currently limited due to their platform-specific nature. Maintaining the code that makes up these solutions is difficult due to dependencies in complicated runtime environments.
Meta AI used AITemplate (AIT) to improve performance up to 12x on NVIDIA GPUs and 4x on AMD GPUs compared with eager mode within PyTorch. The AITemplate (AIT) system consists of a front-end layer that performs various graph transformations and a back-end layer producing C++ kernel templates for the GPU target. The company stated that the vision behind the framework is to support high-speed while maintaining simplicity.
Additionally, AITemplate (AIT) offers Tensor Core (NVIDIA GPU) and Matrix Core (AMD GPU) performance that is almost identical to hardware native on popular AI models like transformers, convolutional neural networks, and diffusers. The A100 and MI200 GPU systems from NVIDIA and AMD, which are both often utilized in data centers for research institutions, IT firms, and cloud computing service providers, among others, currently have AITemplate (AIT) enabled.
AITemplate (AIT) includes advanced transformer block optimizations and enhanced kernel fusion, which is an optimization technique that combines multiple kernels into a single kernel to operate them more effectively. AIT also restricts the time it spends visiting outside libraries.
Meta AI aims to provide methods that can address environmental issues by reducing carbon emissions in addition to expanding the range of platforms that are available for AI. Studies have shown that the use of GPUs can affect carbon emissions. GPU execution is accelerated by AITemplate (AIT), which can reduce emissions even more.
Meta also plans to extend AITemplate (AIT) to additional hardware systems, such as Apple M-series GPUs, as well as CPUs from other technology providers.