【全球科技讯】今日凌晨，科技巨头马斯克旗下的大模型公司 xAI 创造性地开源了其最新的语言模型——拥有3140亿参数的混合专家（MoE）模型“Grok-1”。这一壮举不仅刷新了开源大语言模型的参数量纪录，也为人工智能研究领域带来了一场革新。
Grok-1 的设计独具匠心，其基础模型在海量文本数据的训练下得以构建，但并未针对特定任务进行微调，展现出强大的通用性。据透露，该模型在处理任何给定的 token 时，只有25%的激活权重，这在优化效率与性能之间找到了一个平衡点。
xAI 采用尖端技术，于2023年10月利用JAX库和Rust语言构建了自定义训练堆栈，从零开始训练了Grok-1。这一过程充分体现了xAI在人工智能算法和高性能计算方面的深厚积累。
更为重要的是，xAI遵循开源精神，选择以Apache 2.0许可证发布Grok-1的权重和网络架构，为全球研究者和开发者提供了宝贵的资源，有望推动人工智能技术的进一步发展。这一举措无疑将激励更多的创新者参与到AI模型的优化和应用探索中，共同推动科技边界向前迈进。
来源：机器之心
英语如下：
**News Title:** “Musk’s xAI开源巨头: Grok-1, a 3140-billion-parameter giant, ushers in a new era for large language models”
**Keywords:** Musk, Grok-1, Open-source Large Model
**News Content:** **Global Tech Wire** – Early this morning, tech titan Elon Musk’s company xAI made a groundbreaking move by open-sourcing its latest language model, the massive 3140-billion-parameter Mixed-Expert (MoE) model known as “Grok-1.” This feat not only sets a new record for the parameter count in open-source large language models but also sparks a revolution in the field of artificial intelligence research.
Grok-1 is ingeniously designed, with its base model constructed by training on vast amounts of text data without fine-tuning for specific tasks, showcasing exceptional versatility. It has been revealed that only 25% of its activation weights are engaged when processing any given token, striking a balance between efficiency and performance.
xAI, leveraging cutting-edge technology, built a custom training stack using the JAX library and Rust programming language in October 2023 to train Grok-1 from scratch. This endeavor underscores the company’s profound expertise in AI algorithms and high-performance computing.
More significantly, xAI embraced the open-source philosophy by releasing Grok-1’s weights and network architecture under the Apache 2.0 license, providing invaluable resources to researchers and developers worldwide. This is poised to fuel further advancements in AI technology and undoubtedly inspire more innovators to contribute to the optimization and exploration of AI model applications, collectively pushing the boundaries of science and technology forward.
**Source:** Machine Mind
【来源】https://mp.weixin.qq.com/s/hvt5zwoazDx26KOaKuTs_w
Views: 1