Meta needs 10x more computing power to train Llama 4
Meta, the tech giant behind the advanced open-source language model Llama, has projected a substantial surge in computing power needs for its future models. During the company's Q2 earnings call, CEO Mark Zuckerberg revealed that training Llama 4 will demand nearly 10 times the computing power used for its predecessor, Llama 3. "The amount of computing needed to train Llama 4 will likely be almost 10 times more than what we used to train Llama 3," he said.
Zuckerberg emphasizes proactive capacity building
Zuckerberg emphasized the importance of proactively building capacity, to avoid falling behind due to the extensive lead times associated with initiating new inference projects. This statement comes after Meta's recent launch of an upgraded version of Llama 3, known as Llama 3.1 405B. The updated model boasts a remarkable 405 billion parameters, making it Meta's largest open-source model to date.
Meta plans data center expansion for future AI models
Meta's CFO, Susan Li, disclosed during the earnings call that the company is contemplating various data center projects and capacity expansion to train future AI models. This anticipated growth is projected to escalate capital expenditures in 2025. The disclosure follows a significant increase in Meta's capital expenditures by almost 33% to $8.5 billion in Q2 2024 from $6.4 billion a year earlier. The investments were primarily made in servers, data centers, and network infrastructure.
Flexible approach to AI training capacity
Li further explained Meta's approach to scaling generative AI training capacity, stating that they aim to build their infrastructure flexibly over time. "This will allow us to direct training capacity to gen AI inference or to our core ranking and recommendation work when we expect that doing so would be more valuable," she said.
India is the largest market for Meta AI
During the call, Meta also shared insights about its consumer-facing Meta AI, revealing that India is the largest market for its chatbot. However, Li clarified that the company does not expect Gen AI products to significantly contribute to revenue. This information provides a glimpse into Meta's global user base and its strategic focus on markets with high potential for AI adoption.
