【全球科技讯】今日凌晨，科技巨头马斯克麾下的大模型公司 xAI 传来重大消息，该公司正式开源了其最新的、拥有3140亿参数的混合专家（MoE）模型——Grok-1。这一举动标志着Grok-1成为了目前参数量最大的公开大语言模型，无疑将为人工智能研究领域带来深远影响。
据机器之心报道，Grok-1的基础模型是在海量文本数据上训练而成，未经任何特定任务的微调，以保持其泛化能力。该模型的一大特点是，每个token上的激活权重仅为25%，这在保证模型效率的同时，也展示了其在处理复杂语言任务时的高效性能。值得注意的是，Grok-1的训练工作是在2023年10月完成的，xAI采用的是基于JAX库和Rust语言的自定义训练堆栈，从零开始构建这一庞大而精细的模型。
xAI遵循开源精神，按照Apache 2.0许可证对外公开Grok-1的权重和网络架构，这一举措将为全球的科研人员和开发者提供宝贵的资源，推动人工智能技术的进一步创新和发展。开源Grok-1的决定，无疑将加速大模型在各行业的应用，同时也预示着人工智能在理解与生成自然语言方面的潜力将迎来新的突破。
英语如下：
**News Title:** “Musk’s xAI开源巨头: Grok-1, a 3140-billion-parameter Pioneering Large Language Model”
**Keywords:** Musk, Grok-1, Open-Source Large Model
**News Content:** **Global Tech Update:** Early this morning, a major announcement came from tech giant Elon Musk’s big model company, xAI, revealing that it has officially open-sourced its latest Mixed-Expert (MoE) model, Grok-1, boasting an unprecedented 3140 billion parameters. This move signifies Grok-1 as the largest publicly available large language model to date, poised to significantly impact the field of artificial intelligence research.
According to Machine Mind, Grok-1’s base model was trained on massive amounts of textual data without fine-tuning for any specific task, thereby preserving its generalization capabilities. A key feature is that only 25% of the activation weights are allocated to each token, striking a balance between model efficiency and high performance in tackling complex language tasks. Notably, Grok-1’s training was completed in October 2023, with xAI utilizing a custom training stack built from scratch, leveraging the JAX library and Rust programming language.
Adhering to the open-source spirit, xAI is releasing Grok-1’s weights and network architecture under the Apache 2.0 license, providing invaluable resources to researchers and developers worldwide and fueling further innovation in AI technology. The decision to open-source Grok-1 is expected to expedite the adoption of large models across industries and foreshadows new breakthroughs in AI’s potential for understanding and generating natural language.
【来源】https://mp.weixin.qq.com/s/hvt5zwoazDx26KOaKuTs_w
Views: 1