Meta's new AI chips work faster than before

Meta promises that its next generation of custom AI chips will be more powerful and able to train its classification models much faster.

Meta Training and Inference Accelerator (MTIA) is designed to work best with Meta's ranking and recommendation models. Chips can help make training more efficient and inference — also known as the actual inference task — easier.

MTIA is a big part of its long-term plan to build infrastructure around how AI is used in its services, the company said in a blog post. It wants to design its chips to work with current technology infrastructure and future developments in graphics processing units.

“Meeting our ambitions for our custom silicon means investing not only in compute silicon but also in memory bandwidth, networking, and capacity as well as other next-generation hardware platforms,” Meta said in her post.

Meta announced MTIA v1 in May 2023, with a focus on making these chips available to data centers. The next-generation MTIA chip will also likely target data centers. MTIA v1 was not expected to be released until 2025, but Meta said two MTIA chips are now in production.

Right now, MTIA mainly trains classification and recommendation algorithms, but Meta said the goal is to eventually expand the chip's capabilities to begin training generative AI such as llama language models.

Meta said the new MTIA chip “is primarily focused on providing the right balance between compute, memory bandwidth and memory capacity.” This chip will have 256MB of on-chip memory at 1.3GHz compared to the first version's 128MB of 800GHz memory. Early testing results from Meta showed that the new chip performed three times better than the first-generation version across four models that the company evaluated.

Competition to purchase powerful chips has emphasized the need for dedicated chips to power AI models. Demand for chips has grown so much that Nvidia, which currently dominates the AI ​​chip market, is worth $2 trillion.