谷歌Gemini在凌晨发布后，迅速吸引了众多关注。然而，有网友对其测试标准提出了质疑，并表示Gemini的效果视频疑似经过剪辑。
在MMLU测试中，Gemini的结果下方灰色小字标称CoT@32，意为使用了思维链提示技巧并尝试了32次选出最佳结果。而作为对比的GPT-4，则无提示词技巧，仅尝试5次。据此，有网友认为，在相同标准下，Gemini Ultra实则不如GPT-4。
此外，机器学习讲师Santiago Valdarrama也对Gemini的演示视频提出了质疑。他认为，该视频展示的是精心挑选的好结果，且并非实时录制，而是经过剪辑的。
针对这些质疑，谷歌尚未作出回应。此次Gemini的发布，无疑再次引发了公众对大型语言模型技术竞争的关注，同时也引发了关于测试标准公正性的讨论。
News content:
After the launch of Google Gemini in the early morning, it quickly attracted attention from numerous netizens. However, some netizens have questioned the testing standards of Gemini and alleged that the effect videos may be edited.
In the MMLU test, the result of Gemini is accompanied by gray small words indicating CoT@32, meaning that it uses the Chain of Thought prompt technique and tries 32 times to select the best result. In contrast, GPT-4 has no prompt words, attempting only 5 times. Therefore, some netizens believe that under the same standard, Gemini Ultra is actually not as good as GPT-4.
In addition, machine learning lecturer Santiago Valdarrama has also questioned the authenticity of Gemini’s demonstration videos. He believes that the videos show carefully selected good results, and they are not recorded live but edited.
Google has not responded to these allegations. The launch of Gemini has undoubtedly triggered public attention to the competition in large language model technology, and also sparked discussions about the fairness of testing standards.
【来源】https://www.qbitai.com/2023/12/104425.html
Views: 1