上海人工智能实验室近日迈出了重要的一步，正式开源其精心打造的高质量语料库——“万卷CC”（WanJuan-CC）。这一大规模的预训练语料库涵盖了过去十年间互联网上的公开内容，总计包含1千亿字符，约400GB的丰富英文数据，显示出对人工智能研究与开发的深度支持。
“万卷CC”作为“大模型语料数据联盟”今年的首推开源项目，旨在为学术界和工业界提供强有力的数据基础，以促进更加智能且可靠的AI大模型的研发。这一开源举措将极大地推动AI技术的进步，为模型训练提供更为精准和全面的素材，同时也有助于降低相关研究和应用的门槛，鼓励更多创新者参与到AI领域。
上海AI实验室的这一行动彰显了其在推动人工智能开源生态建设上的决心和远见。通过开放如此庞大的数据集，他们不仅贡献了自身的技术积累，也为全球的科研人员和开发者打造了一个共享知识和智慧的平台。这不仅将加速AI技术的迭代，也将对未来的智能应用产生深远影响。
英语如下：
**News Title:** “Shanghai AI Lab ReleasesMassive Open-Source Corpus ‘WanJuan CC’: A 100B Character Milestone for AI Large Models”
**Keywords:** Shanghai AI Lab, WanJuan CC, Open-Source Corpus
**News Content:**
The Shanghai Artificial Intelligence Laboratory has taken a significant stride forward by officially releasing its meticulously crafted high-quality corpus, the “WanJuan CC” (WanJuan-CC). This vast pre-training corpus encompasses public content from the internet over the past decade, amounting to a staggering 100 billion characters, or approximately 400 GB of rich English data, demonstrating strong support for AI research and development.
“WanJuan CC,” as the first open-source project of the “Large Model Corpus Data Alliance” this year, aims to provide a robust data foundation for academia and industry, fostering the development of more intelligent and reliable AI large models. This open-source initiative will significantly advance AI technology by offering more precise and comprehensive training materials, while also lowering the barrier to entry for related research and applications, encouraging more innovators to engage in the AI field.
By making such a massive dataset available, the Shanghai AI Lab showcases its commitment and foresight in promoting the growth of an open-source AI ecosystem. By contributing their technological accumulation, they are creating a platform for global researchers and developers to share knowledge and insights. This will not only accelerate AI technology iterations but also have a profound impact on future intelligent applications.
【来源】https://mp.weixin.qq.com/s/Pt02LXlh2Uu_hgM0ZL5GGg
Views: 1