中国科学技术大学特任副研究员王皓在InfoQ的采访中表示，当前推荐大模型在应用中面临多项挑战，包括数据规模大、模型复杂性高、增量处理难等。为了应对这些挑战，王皓及其团队采用了数据并行、流水线并行、张量并行等技术进行加速，并对模型架构进行了创新，研究了基于Mamba等状态空间模型的推荐大模型架构，解决了Transformer架构的自注意力机制计算和存储复杂度的问题。此外，他们还引入了多行为、跨域数据，更准确地捕捉用户的兴趣动态，挖掘更加全面和细致的用户画像。在通用大模型的发展方面，王皓认为它们在设计上为推荐大模型提供了启示，特别是在文本作为Token的通用性方面。通过这些努力，推荐大模型正逐渐克服其在应用中的局限性，向着更加高效和精准的方向发展。
英语如下：
News Title: “USTC Expert Unveils: Big Model Recommendation Systems Face Five Major Challenges”
Keywords: Recommendation Big Models, Data Quality, Technical Challenges
News Content: Dr. Hao Wang, an adjunct research fellow at the University of Science and Technology of China, stated in an interview with InfoQ that current big model recommendation systems face numerous challenges in application, including large data scales, high model complexity, and difficulties in incremental processing. To address these challenges, Wang and his team have adopted technologies such as data parallelism, pipeline parallelism, and tensor parallelism for acceleration, and have innovated in model architecture, studying the recommendation big model architectures based on state space models like Mamba, solving the computational and storage complexity issues of the Transformer architecture’s self-attention mechanism. Additionally, they have introduced multi-behavior and cross-domain data to more accurately capture user interest dynamics and delve into more comprehensive and detailed user profiles. In the development of general big models, Wang believes they provide inspiration for recommendation big models, particularly in the generality of text as Tokens. Through these efforts, recommendation big models are gradually overcoming their limitations in application and are developing towards more efficient and accurate directions.
【来源】https://mp.weixin.qq.com/s/UPiOJOifh0ygIaHMu3QLog
Views: 3