Xiaomi Is Building A GPU Cluster and Will Heavily Invest in AI Large Models

Jiemian News has exclusively learned that Xiaomi is currently building its own GPU cluster and will heavily invest in AI large models. Xiaomi‘s large model team already has 6,500 GPU resources at the time of establishment.

Jiemian News sought confirmation from Xiaomi on this matter, but as of press time, Xiaomi has not commented.

An informed source told Interface News reporters that the plan has been implemented for several months, with Lei Jun playing an important leadership role. ‘In terms of AI hardware, the most crucial aspect is smartphones rather than glasses. It is impossible for Xiaomi not to go all-in in this field.’

Xiaomi‘s emphasis on AI large models had shown signals earlier. On December 20th, according to First Financial Daily report, one of the key developers of DeepSeek open-source large model DeepSeek-V2, Luo Fuli, will join Xiaomi or work at Xiaomi‘s AI Lab to lead the Xiaomi large model team.

Previously, an important innovation in the model architecture of DeepSeekV2 was the adoption of MLA (Multi-head Latent Attention), a technology that played a key role in reducing the cost of using large models, and Luo Fuli was one of the core figures in this work.

In April 2023, Xiaomi AI Lab’s large model team was officially formed, with Luan Jian appointed as the head of the large model team, reporting to Wang Bin, Vice Chairman of Xiaomi Technical Committee and Director of AI Lab.

Luan Jian previously served as the head of the AI Lab’s speech generation team and held positions such as researcher at Toshiba (China) Research Institute, senior speech scientist at Microsoft (China) Engineering Institute, chief speech scientist and head of speech team for Microsoft Xiaoice.

SEE ALSO: Xiaomi’s AI Large Model MiLM Officially Passes the Record

At the same time, Lei Jun wrote about his views on large models and AIGC. He mentioned that Xiaomi has been working in AI field for many years with teams like AI Lab, Xiao Ai voice assistant, autonomous driving etc. ‘Regarding large models, we will certainly go all out and embrace them firmly. We are developing some interesting technologies and products. Once we polish them up well enough, we will showcase them.’

Subsequently during Xiaomi‘s 2023 anniversary speech event , Lei Jun once again talked about progress made by company’s big model business.

He said that after the team was established, Xiaomi‘s main breakthrough direction in large-scale model technology is lightweight and local deployment. As the latest achievement, Xiaomi has initially run a large-scale model on the mobile side (with 1.3 billion parameters), with effects in some scenarios approaching those of cloud-based models with 6 billion parameters, and will simultaneously push an upgraded version of Xiao Ai voice assistant.

At that time, Xiaomi had two parameter-level models: MiLM-6B/1.3B. Wang Bin emphasized in interviews with media such as Jiemian News that including data and algorithms, all models trained by Xiaomi are built from scratch. However, the team does not oppose third-party large models and will combine self-research with third-party cooperation to promote its development of large models.

It is worth noting that when Xiao Ai voice assistant was first upgraded, a hybrid solution combining third-party and self-developed approaches was used for the large model version.

Public information shows that since establishing the AI team in 2016, Xiaomi‘s artificial intelligence team has expanded seven times over six years. The size of personnel in related fields has exceeded 3,000 people; their AI technical capabilities cover areas such as vision, acoustics, speech recognition, NLP (Natural Language Processing), knowledge graphs, machine learning, large-scale models,and multimodal directions; gradually integrating into business sectors such as smartphones,cars,AIoT(AIoT),robots,and more.

Report