**ICML 2024研究揭示：Transformer在数学推理中如何抉择——基于样例还是基于规则**
近日，北京大学物理学院的胡逸在即将加入人工智能研究院读博之际，与其导师张牧涵助理教授团队共同发表论文，探究Transformer在处理数学推理问题时的方式。论文指出，针对长整数加法等任务，Transformer更倾向于依赖训练语料库中相似的样例而非固定的数学规则。这一发现已在ICML 2024会议上引起广泛关注。
随着人工智能技术的飞速发展，大语言模型（LLMs）在各种任务中表现出惊人的性能。但在处理某些看似简单的数学推理问题时，它们却常常捉襟见肘。论文通过深入研究发现，Transformer在处理这类问题时，更倾向于借鉴过去的案例而非遵循固定的数学规则。这一现象在人类的数学学习中也出现过，但LLMs如何做到这一点仍然是一个待解之谜。张牧涵团队此次研究不仅为我们提供了关于Transformer如何处理数学推理的新视角，也为未来的模型优化提供了新的思路。随着研究的深入，我们期待人工智能在各个领域展现更加强大的能力。
英语如下：
News Title: “The Mystery of Transformer’s Mathematical Reasoning: Example-based or Rule-based?”
Keywords: 1. Transformer reasoning mechanism
News Content: **ICML 2024 Research Reveals: How Does Transformer Decide in Mathematical Reasoning – Example-based or Rule-based**
Recently, Hu Yi from the Physics School of Peking University, on the eve of joining the Artificial Intelligence Institute to pursue his PhD, co-authored a paper with his advisor, Assistant Professor Zhang Muhan, exploring how Transformer handles mathematical reasoning problems. The paper indicates that when faced with tasks such as long integer addition, Transformer tends to rely more on similar examples from the training corpus rather than fixed mathematical rules. This discovery has garnered widespread attention at the ICML 2024 conference.
With the rapid development of artificial intelligence technology, large language models (LLMs) have demonstrated impressive performance in various tasks. However, when dealing with certain seemingly simple mathematical reasoning problems, they often struggle. The paper delves into the research and finds that when facing such problems, Transformer tends to draw on past cases rather than follow fixed mathematical rules. This phenomenon has also occurred in human mathematical learning, but how LLMs do it remains a mystery. Zhang Muhan’s team’s research not only provides a new perspective on how Transformer handles mathematical reasoning but also offers new ideas for future model optimization. With further research, we look forward to artificial intelligence demonstrating stronger capabilities in various fields.
【来源】https://www.jiqizhixin.com/articles/2024-06-29
Views: 1