近日，谷歌DeepMind团队宣布推出两款新型基础模型——Hawk和Griffin。这两款模型的核心创新是RG-LRU层，这是一种全新的门控线性循环层技术，被设计用来替代传统的多查询注意力（MQA）。RG-LRU层通过引入循环块，提高了模型的计算效率和性能。Hawk模型结合了多层感知机（MLP）和循环块，而Griffin在此基础上还整合了局部注意力机制，以增强模型的感知能力。这一突破性的研究成果，有望在自然语言处理、图像识别等多个AI领域产生广泛影响。
Title: Google DeepMind Unveils New AI Models
Keywords: AI Models, Recurrent Layer, Attention Mechanism
News content:
In recent days, the Google DeepMind team has announced the launch of two new fundamental models – Hawk and Griffin. The core innovation behind these models is the RG-LRU layer, a novel gated linear recurrent layer technology designed to replace the traditional multi-query attention (MQA). By introducing a recurrent block, the RG-LRU layer increases the computational efficiency and performance of the models. The Hawk model combines multi-layer perceptrons (MLP) and recurrent blocks, while the Griffin model adds local attention mechanisms on top of this, enhancing the model’s perception capabilities. This breakthrough research could have far-reaching impacts across various AI fields, including natural language processing and image recognition.
【来源】https://mp.weixin.qq.com/s/RtAZiEzjRWgqQw3yu3lvcg
Views: 1