Meta has detailed MTIA 300, its first in-house accelerator optimized for training ranking and recommendation models.
Unlike large-language-model training, where raw floating-point throughput is often the dominant concern, Meta says recommendation models spend much more time communicating between accelerators. MTIA 300 addresses that by integrating networking and collective communication directly into the chip.
The communication pressure comes largely from embedding tables, which Meta says can contain more than 99% of a recommendation model’s parameters. Training these models across hundreds of accelerators generates frequent AllReduce, AllToAll, and AllGather operations, creating a workload where the network can become as important as the compute itself.
Meta first moved the network interface into the accelerator package. MTIA 300 contains two network chiplets with six custom 800 Gbps RDMA NICs each, providing 1.2 TB/s of total I/O bandwidth without crossing a PCIe bus. The same twelve NICs support both scale-up communication within a rack and scale-out traffic between racks, allowing Meta to change how bandwidth is divided without redesigning the chip.
Source: Meta
Moving the NICs closer to compute did not remove another source of contention: on conventional GPUs, collective communication can consume the same processing resources needed for training. MTIA 300 instead includes 16 dedicated message engines that handle communication independently of its main compute grid, including near-memory hardware for reduction operations.
Meta reports that this separation allows large matrix operations and collective communication to run concurrently with less than 0.5% degradation in compute throughput. On the GPU architecture used for comparison, Meta measured degradation of more than 20% when the two workloads overlapped.
The hardware was co-designed with HCCL, Meta’s collective-communication library. Rather than having the host CPU orchestrate every communication operation while a job runs, HCCL compiles collective operations into subgraphs that MTIA 300’s message engines can execute autonomously. Once those instructions reach the accelerator, the host is no longer involved in driving the communication.
In production, Meta says HCCL reaches up to 940 GB/s of communication bandwidth within a rack. On a 150-billion-parameter recommendation model running across 40 accelerators, the company reports that MTIA 300 reduced total communication time by 3.9x compared with an equivalent GPU cluster.
MTIA 300 also marks an expansion of Meta’s broader custom-silicon programme. The company says it already operates hundreds of thousands of MTIA accelerators for inference and plans four further generations over the next two years, spanning ranking, recommendation, and generative-AI workloads. Meta has also expanded its partnership with Broadcom to co-develop future MTIA generations, while continuing to source accelerators from AMD, NVIDIA, and other vendors as part of what it calls a portfolio strategy.
That strategy reflects a wider shift among hyperscalers toward workload-specific AI silicon. Google has continued expanding its TPU programme, Amazon has grown its Trainium business, and Microsoft is developing successive generations of Maia accelerators as cloud providers look for alternatives to relying exclusively on general-purpose GPUs.
MTIA 300 shows how far Meta is prepared to take that specialization. Rather than treating networking as supporting infrastructure around the accelerator, the company designed compute, communication hardware, and collective software together around a workload where moving data can be as important as processing it.