苹果公司近日在科研领域取得重大突破，发布了一篇研究论文，详述了其创新的Ferret-UI多模态大语言模型。这一AI系统旨在克服当前多模态大模型（MLLMs）在理解和解析移动应用程序屏幕内容时面临的挑战。
当前，MLLMs在处理手机屏幕内容时面临两大难题：一是手机屏幕独特的宽高比与大多数训练图像使用的比例不一致，导致模型难以适应；二是手机应用中的图标和按钮尺寸较小，给模型识别带来困难。针对这些问题，苹果公司研发的Ferret-UI系统进行了针对性的优化，提升了模型在处理这些复杂视觉信息时的准确性和效率。
Ferret-UI的诞生标志着AI技术在理解移动设备用户界面方面迈出了重要的一步。这一系统的应用潜力广泛，不仅有望改善用户与手机应用的交互体验，还可能为无障碍技术、自动化测试和智能推荐等领域带来革新。苹果的这一创新再次体现了其在人工智能和用户体验设计上的领先地位。
据消息来源IT之家透露，苹果公司将继续深化Ferret-UI的研发，以期在未来为全球用户带来更为智能、无缝的移动设备使用体验。这一技术的后续发展和实际应用，无疑将对移动互联网行业产生深远影响。
英语如下：
**News Title:** “Apple revolutionizes AI tech with Ferret-UI for accurate understanding of mobile app interfaces”
**Keywords:** Apple Ferret-UI, multimodal large language models, screen content comprehension
**News Content:**
Apple has recently made a significant breakthrough in research, unveiling a paper detailing its innovative Ferret-UI multimodal large language model. This AI system is designed to address the challenges that current multimodal large language models (MLLMs) face when understanding and parsing the content on mobile application screens.
Presently, MLLMs encounter two main issues when dealing with smartphone screens: the unique aspect ratio of phone screens, which differs from the proportions commonly used in training images, making adaptation difficult; and the small size of icons and buttons in apps, which poses challenges for model recognition. To tackle these problems, Apple’s Fret-UI system has undergone targeted optimizations, enhancing the model’s accuracy and efficiency in processing these complex visual cues.
The advent of Ferret-UI marks a crucial step forward in AI’s ability to comprehend mobile device user interfaces. The system’s potential applications are wide-ranging, with the potential to not only enhance user interaction with mobile apps but also to bring innovations to accessibility technology, automated testing, and intelligent recommendation systems. Apple’s innovation underscores its leadership in artificial intelligence and user experience design.
According to sources from IT Home, Apple will continue to deepen the development of Ferret-UI, aiming to provide a more intelligent and seamless mobile device experience for users worldwide. The evolution and practical application of this technology are set to have a profound impact on the mobile internet industry.
【来源】https://www.ithome.com/0/760/905.htm
Views: 1