喵~ 喵喵，大家好呀！今天有个超级有趣的消息要告诉你们哦！智源研究院，就是那个很厉害的科研机构，它们发布了一个叫CMMU的东西，全称是中文多模态多题型理解及推理评测基准。这可不是普通的考试哦，是专为像GPT-4V这样的智能模型准备的挑战呢！这个CMMU v0.1版本，是从咱们国家的小学到高中的考题里精心挑选出3603道题目，种类多得很，有单选、多选还有填空，保证让模型们大展身手。
不过，最近大名鼎鼎的GPT-4V在CMMU面前似乎遇到了难题，答题准确率只有大约30%喵。虽然这个比例听起来不高，但别忘了，这些题目可都是精心设计，难度不小的。据分析，GPT-4V在图像理解和推理这块还有点小迷糊，需要继续加油学习哦。智源研究院的这个CMMU，就像是智能模型的“升级版补习班”，帮助它们更好地理解和应对复杂的任务。下次再见，希望GPT-4V能带给我们更多的惊喜呢！喵~
英语如下：
News Title: “Zhiyuan Releases CMMU Benchmark, GPT-4V Faced with Chinese Challenge, Achieves Only 30% Accuracy”
Keywords: Zhiyuan CMMU, GPT-4V, Multimodal Evaluation
News Content: Meow~ Meow-meow, hello everyone! I have a super interesting news story for you today! Zhiyuan Institute, that super smart research place, they’ve launched something called CMMU – the Chinese Multimodal Multiple-Task Understanding and Inference Benchmark. It’s not your ordinary test, it’s a challenge tailor-made for AI models like GPT-4V! Version 0.1 of CMMU picked 3,603 questions carefully from Chinese primary to high school exams, with all sorts of types – multiple-choice, multiple-select, and fill-in-the-blanks. It’s a real showcase for these models.
But guess what, the famous GPT-4V seems to have met its match in CMMU, scoring only around 30% accuracy, meow. Even though that sounds low, these questions are cunningly crafted and quite challenging. Analysts say GPT-4V still needs some work on image understanding and reasoning. Zhiyuan’s CMMU is like an ‘advanced study group’ for AI models, helping them better handle complex tasks. Next time we meet, we hope GPT-4V will surprise us with even greater improvements! Meow~
【来源】https://mp.weixin.qq.com/s/wegZvv4hwLef0BpdIh32-A
Views: 1