China develops 100-billion-parameter AI model using domestic tech
China's state-owned telecom giant, China Telecom, has successfully developed a 100-billion-parameter artificial intelligence (AI) model, named TeleChat2-115B. This achievement is significant as it was accomplished solely using China's domestic computing power. The project was revealed in a GitHub update last month, which stated that the model was "trained entirely with domestic computing power and open sourced."
TeleChat2-115B: A product of 10 trillion tokens
The TeleChat2-115B model was trained using a massive dataset of 10 trillion tokens, derived from high-quality Chinese and English texts. The project's GitHub page provided insights into the potential training method used by China Telecom. It suggested compatibility with the "Ascend Atlas 800T A2 training server," a product developed by Huawei that supports Kunpeng processors.
Huawei's processors and their role in AI training
The Kunpeng 920 7265 and Kunpeng 920 5250 processors, supported by the Ascend Atlas server, are key components in this project. These processors operate at a speed of up to 3.0GHz with 64 cores and up to 2.6GHz with 48 cores, respectively. They are built on the Arm v8 architecture using a sophisticated 7nm process.
TeleChat2-115B's parameter count and AI model comparison
With 100 billion parameters, TeleChat2-115B falls behind some recent models such as Llama models with around 400 billion parameters, and OpenAI's o1 estimated to have been trained with 200 billion parameters. However, it's important to note that a model's effectiveness is not solely determined by its parameter count. The lower parameter count of TeleChat2-115B implies that its training likely required less computing power than other projects.
China's AI ambitions undeterred by tech export sanctions
The development of TeleChat2-115B indicates that China's pursuit of AI advancements remains unaffected by US's tech export sanctions. Despite not having access to the most advanced AI hardware, China Telecom has demonstrated its ability to harness significant power. This is largely due to its vast scale, with revenues exceeding $70 billion and providing services for over half a billion wired and wireless subscriptions.
