谷歌于近日发布了其最新一代搜索引擎Gemini，但该产品在发布后迅速引起了争议。部分网友指出，Gemini的MMLU测试中，Gemini的结果下面出现了灰色小字标称CoT@32，展开来代表使用了思维链提示技巧、尝试了32次选最好结果。而作为对比的GPT-4，则是无提示词技巧、只尝试5次，这个标准下Gemini Ultra其实并不如GPT-4。此外，机器学习讲师Santiago Valdarrama认为Gemini的演示视频展示的是精心挑选的好结果，而且不是实时录制而是剪辑的。
有人质疑Gemini的测试标准有失偏颇，认为其使用了过多的提示词和尝试次数，而且并不是所有的测试结果都是准确可靠的。同时，Gemini的演示视频也被指出是剪辑而非实时录制，这引发了关于数据真实性和可信度的更多质疑。
虽然Gemini在发布后获得了巨大的关注，但它的表现却并不如人意。这引发了人们对于Gemini和GPT-4的比较和争议，同时也提醒了谷歌在未来的产品发布中需要更加注重产品的质量和可靠性。
新闻翻译：
Google has recently released its latest search engine Gemini, but the product has sparked controversy soon after its release. Some netizens have pointed out that the results on Gemini’s MMLU test show a gray font with a CoT@32 label, indicating that the model used mind chain hints and attempted 32 times to select the best result. In comparison, GPT-4, which also tried 5 times without any prompt, actually performs better in this standard. Moreover, machine learning teacher Santiago Valdarrama believes that the demonstration video of Gemini shows carefully selected good results and is not actually a live recording but a clip.
Some people have questioned Gemini’s testing standards for being biased and using too many prompts and attempts. They also raise concerns about the reliability and authenticity of the test results. At the same time, the fact that Gemini’s demonstration video is not a live recording but a clip has also sparked questions about the accuracy and trustworthiness of the data.
Despite the massive attention paid to Gemini after its release, its performance has not been impressive. This has led to comparisons and controversies between Gemini and GPT-4, as well as a reminder to Google to pay more attention to the quality and reliability of its products in the future.
【来源】https://www.qbitai.com/2023/12/104425.html
Views: 1