近日，香港中文大学贾佳亚团队与MIT合作发布了全球首个70B长文本大语言模型。该模型可生成超过100,000个token，是同类模型的前几名。该模型可以生成两行代码，在一台8卡A100机器上运行，成本低廉。该研究团队还基于LongLoRA技术，发布了全球首个拥有70B参数量的长文本对话大语言模型LongAlpaca。目前，LongLoRA技术和LongAlpaca已开源。
该模型的文本长度拓展技术LongLoRA使得70B模型的文本长度扩展到32k tokens，而70B模型的文本长度扩展到100k tokens。该技术可望在自然语言处理、文本生成等领域广泛应用。
该研究团队表示，未来将继续优化LongLoRA技术，并探索更多应用场景。
新闻翻译：
Title: Global’s first 70B long text大 language model is launched
Keywords: Global’s first, 70B long text, large language model
News content:
Recently, the joint team of the Chinese University of Hong Kong and MIT has announced the launch of the global’s first 70B long text large language model. This model can generate more than 100,000 tokens and is among the top few in the same category. It can be run on two lines of code and a single 8-card A100 machine, making it cost-effective. The research team has also released the global’s first long text dialogue large language model with 70B parameters, LongAlpaca, based on the LongLoRA technology. Both LongLoRA technology and LongAlpaca have now been opened up for public use.
The text length extension technology LongLoRA allows the text length of the 70B model to be extended to 32k tokens, and the text length of the 70B model to be extended to 100k tokens. This technology is expected to be widely applicable in the fields of natural language processing and text generation.
The research team said that they will continue to optimize LongLoRA technology and explore more application scenarios in the future.
【来源】https://www.jiqizhixin.com/articles/2023-10-09
Views: 1