Guangzhou, China – In a significant step towards bringing the power of large language models (LLMs) to edge devices, a collaborative team from the Hong Kong University of Science and Technology (Guangzhou) (HKUST(GZ)), University College London (UCL), and the Institute of Automation, Chinese Academy of Sciences (CASIA) has announced the development of PLM (Peripheral Language Model). This innovative model, designed with hardware-software co-optimization, aims to overcome the memory, compute, and I/O bottlenecks that have traditionally hindered the deployment of LLMs on resource-constrained devices like smartphones and embedded systems.
The research, led by Professor M. N. Ng, President of HKUST(GZ), Professor Lei Chen, Dean of the Information Hub at HKUST(GZ), and Professor Jun Wang of the UCL AI Centre, tackles the critical challenge of enabling AI with billions of parameters to operate efficiently on edge devices, paving the way for ubiquitous AI applications.
The ability to run sophisticated AI models directly on edge devices is crucial for realizing the full potential of the Internet of Things, explains Deng Cheng, the first author of the paper and a research assistant at HKUST(GZ). PLM represents a significant advance in addressing the limitations of memory, compute, and I/O that have previously prevented the deployment of large language models on these platforms.
The core innovation of PLM lies in its architecture, which uniquely combines the MLA (Multi-Level Attention) mechanism with ReLU2-activated Feed Forward Networks (FFNs). This synergistic design is specifically tailored for the constraints of edge devices. The team claims that PLM outperforms existing small language models trained on public datasets.
The research team comprises a diverse group of experts, including Dr. Luoyang Sun and Dr. Yongcheng Zeng from CASIA, Xinjian Wu from UCL, and Qingfa Xiao and Wenxin Zhao, PhD students at HKUST(GZ). Postdoctoral fellow Jiachuan Wang from HKUST and Assistant Professor (Research) Haoyang Li from Hong Kong Polytechnic University also contributed to the project. The corresponding authors are Dr. Cheng Deng, Professor Lei Chen, and Professor Jun Wang.
The development of PLM signals a promising direction for the future of edge AI, enabling a wide range of applications from personalized mobile experiences to intelligent embedded systems. The team’s hardware-software co-design approach offers a blueprint for overcoming the challenges of deploying increasingly complex AI models on resource-limited devices. Further research and development in this area are expected to accelerate the adoption of AI across various industries and applications.
References:
- Information provided by the PLM team.
- 把MLA和稀疏激活带到端侧！港科大广州和伦敦大学学院团队联合发布软硬协同设计的边缘语言模型PLM. 机器之心 (Machine Heart), March 27, 2025. [Link to original article, if available]
Note: The date in the original article is in the future (2025/03/27). I have retained it as is, assuming it’s a planned release date or a typo. If a specific publication or pre-print server is available, please provide the link for a more complete citation.
Views: 0