Mistral AI近日宣布，其Mistral 7B v0.2基模型已正式开源，该模型支持高达32K的上下文，这一消息是在一场名为“Cerebral Valley”的黑客松活动中公布的。Mistral AI以其“Mistral Tiny”系列模型而闻名，此次开源的Mistral 7B v0.2 Base Model正是Mistral-7B-Instruct-v0.2的原始预训练模型。此次更新主要集中在三个方面：一是将上下文长度从8K扩展到32K；二是引入了Rope Theta，其值设为1e6；三是取消了滑动窗口机制。这一系列改进旨在提升模型的性能和处理大规模数据的能力。
英文标题：Mistral AI Unveils Open Source 7B v0.2 Model with 32K Context Support
英文关键词：Mistral AI, Open Source, Model Update, Cerebral Valley, Hackathon
英文新闻内容：
In a recent announcement, Mistral AI has revealed the open source release of its Mistral 7B v0.2 base model, which now supports a significant 32K context. This news was shared during a hackathon event called “Cerebral Valley,” where the company showcased its latest advancements. The model in question is the foundation behind the Mistral-7B-Instruct-v0.2, part of Mistral AI’s “Mistral Tiny” series. The update includes three key enhancements: extending the context length from 8K to 32K, introducing Rope Theta with a value of 1e6, and removing the sliding window mechanism, all aimed at improving performance and handling of large-scale data.
【来源】https://mp.weixin.qq.com/s/R56Ob5dZjMh1alhMin8DZw
Views: 1