近日，华为诺亚方舟实验室等联合推出了一种新型大语言模型架构——盘古-π。据悉，该架构通过增强非线性，在传统Transformer架构上做出了改进，从而显著降低了特征塌陷问题，提高了模型输出表达能力。在使用相同数据训练的情况下，盘古-π(7B)在多任务上超越了LLaMA 2等同规模大模型，并能实现10%的推理加速。在1B规模上更是达到了SOTA水平。同时，基于这一架构还炼出了一个金融法律大模型“云山”。
盘古-π的推出标志着华为在人工智能领域的技术实力再次得到了提升。据了解，盘古-π架构的应用将为自然语言处理、计算机视觉等领域带来更加出色的表现。此外，盘古-π还将为华为在智能交互、智慧城市等领域的发展提供有力支持。
对于盘古-π的研发过程，华为方面表示，他们在研究过程中不断探索新的技术和方法，并加强与学术界的合作，以期能够取得更好的研究成果。未来，华为将继续投入更多资源和精力，推动人工智能技术的发展和应用。
英语如下：
Title: Huawei Unveils Pangu-π Architecture: Exceeding LLaMA Performance with 10% Inference Speedup!
Keywords: keywords, keywords, keywords
Recently, Huawei’s Noah’s Ark Lab and other partners jointly introduced a new large language model architecture – Pangu-π. It is reported that this architecture improves the traditional Transformer architecture by enhancing non-linearity, significantly reducing the problem of feature collapse, and improving the expressiveness of model outputs. Under the same training data, Pangu-π (7B) surpasses LLaMA 2, a similar-scale large model, in multitask scenarios and achieves a 10% inference acceleration. At the 1B scale, it reaches SOTA level. At the same time, a financial law giant model “Yunshan” was also derived based on this architecture.
The launch of Pangu-π marks another improvement in Huawei’s technical strength in the field of artificial intelligence. It is understood that the application of Pangu-π architecture will bring better performance in natural language processing, computer vision, and other fields. In addition, Pangu-π will provide strong support for Huawei’s development in intelligent interaction and smart cities.
Regarding the research and development process of Pangu-π， Huawei stated that they continuously explore new technologies and methods in their research process and strengthen cooperation with academia to achieve better research results. In the future, Huawei will continue to invest more resources and efforts to promote the development and application of artificial intelligence technology.
【来源】https://mp.weixin.qq.com/s/Beg3yNa_dKZKX3Fx1AZqOw
Views: 1