Meta showcases AI power with custom chips, data centers, supercomputer
What's the story
Meta has been a slow starter in the AI arms race. But with investors closely watching how the company aims to take on its rivals in the AI sphere, the social media giant has been making some moves.
The firm has now shed light on its in-house infrastructure to help with its AI ambitions, revealing custom chips to run AI models.
Context
Why does this story matter?
The AI race is heating up and custom AI chips have emerged as an area of interest for the tech titans. Google has its own processor, while Microsoft is working with AMD to develop a custom chip.
Meta's failure to read the AI game has always been pointed out by its critics. With the new announcements, the company aims to turn that around.
MTIA
Meta's custom chip focuses on inferencing rather than training
Meta's custom AI chip is called Meta Training and Interface Accelerator (MTIA). The Mark Zuckerberg-led company describes it as a part of a "family" of chips targeting AI training and inferencing workloads.
The focus of the current generation of MTIA is inferencing. It is a process through which algorithms trained on large datasets determine whether to show a post in a user's feed.
Reason
Both MTIAs and GPUs will be used for now
Meta initially used GPUs for inferencing tasks. However, their low efficiency made inferencing complicated.
The company's lack of AI infrastructure hampered its progress as product teams came up with AI-powered features for its suite of apps.
That's why the company decided to turn to custom AI chips capable of training and inferencing. It said it would use both MTIA chips and GPUs for inferencing.
The chip
The chip handles 'low-complexity' and 'medium-complexity' models better then GPU
Semiconductor giant Taiwan Semiconductor Manufacturing Company is manufacturing the MTIA chip. According to Meta, it uses only 25 watts of power, significantly increasing the company's efficiency in performance per watts terms.
It uses an open-source chip architecture called RISC-V. The company said MTIA faced issues with high-complexity models but handled "low-complexity" and "medium-complexity" AI models better than a GPU.
MSVP
Meta is developing a custom chip for video processing needs
Meta talked about another custom chip called Meta Scalable Video Processor (MSVP). It is being developed to deal with video transmission and live streaming needs.
The company plans to offload most of its "stable and mature" video processing workloads to MSVP. In the future, it will support the efficient delivery of generative AI, AR/VR, and other metaverse content.
Data centers
AI-optimized data centers are also in the works
Meta also provided an update on its next-generation data centers. According to the company, the new data centers will have an AI-optimized design with support for liquid-cooled AI hardware and high-performance AI networks.
The firm will start building the first such facility this year. Meta said its new data centers will be faster and more cost-effective to build.
Supercomputer
The final buildout of its supercomputer is done
Meta has completed the final buildout of its AI supercomputer called Research SuperCluster (RSC). It features 16,000 GPUs and was built to train large AI models.
The company used RSC to train LLaMA (Large Language Model Meta AI). According to Meta, LLaMA contains 65 billion parameters and is trained on 1.4 trillion tokens.
In comparison, Google's PaLM 2 is trained on 3.6 trillion tokens.