In a groundbreaking advancement that could revolutionize both home service robots and industrial manufacturing, a team of researchers at Tsinghua University has developed a sophisticated multimodal tactile sensing system. This system, designed to integrate a range of tactile perceptions and visual inputs, aims to create a more efficient and intelligent decision-making brain for robots.
Multifunctional Tactile Sensing Sensor
Led by Professor Zhu Rong, the Tsinghua team has created a sensor that integrates multiple tactile functions, including slip, material, texture, pressure, and temperature sensing, all in one device. This multimodal tactile sensor also fuses with visual inputs to enhance the robot’s perception and decision-making capabilities.
One of the most significant aspects of this research is the inclusion of slip detection, which has been largely overlooked in previous studies. The team achieved an ultra-sensitive slip detection with a response time of just 4ms at a speed of 0.05mm/s, providing robots with the ability to perform delicate operations with enhanced precision.
Enhanced Grasping and Manipulation
The rich tactile perception functions offer technical support for robots to grasp objects more flexibly and perform intricate operations. The team’s innovative grasping strategy combines ultra-fast response and ultra-sensitive slip feedback control mechanisms, ensuring that robots can grip objects with minimal force, thereby avoiding damage to fragile or soft items.
Professor Zhu Rong explained, If a robot lacks slip sensing, it is difficult to determine whether the grip on an object is stable. By equipping robots with slip sensing, especially with ultra-sensitive and ultra-fast responses, we achieve a universal stability in gripping various objects, marking a breakthrough in tactile perception technology.
Integration of Vision and Tactile Sensing
Building on the development of the multimodal tactile sensor, the researchers also proposed a fusion strategy that combines visual and tactile inputs. This integration has led to the establishment of a robot system with a unified architecture for perception, decision-making, and execution, enabling the robot to plan and perform complex tasks autonomously.
For instance, when robots need to pick up small or lightweight items from a surface, such as pens and paper, they demonstrate human-like intelligence by adopting a strategy of moving the items to the edge of the table before picking them up, mimicking the dexterity of human actions.
In another scenario, the robot can sense the presence of liquid in a paper cup through its tactile perception system and then carefully pour the liquid into a container and place the empty cup in a recycling bin.
Research Paper and Application Potential
The team’s research, titled Multimodal tactile sensing fused with vision for dexterous robotic housekeeping, was recently published in Nature Communications. PhD candidates Mao Qian and Liao Zijian are the co-first authors, with Professor Zhu Rong serving as the corresponding author.
The technology holds significant potential for application in both home service and industrial manufacturing sectors.
Home Service
The research demonstrates the potential of the robot system in home service tasks such as sorting items and serving tea. There is a growing demand for companionship and life skill support among the elderly and people with disabilities, but no commercial home service robots have emerged globally.
Robots currently cannot perform home services; they need a rational design that integrates perception, decision-making, and execution, Professor Zhu Rong noted.
Industrial Manufacturing
While industrial robots are already in use in the manufacturing sector, their fine manipulation capabilities in complex tasks like assembly, quality inspection, and sorting are limited. The team’s multimodal perception decision system shows promise in enhancing the precision of industrial automation, potentially boosting the level, quality, and efficiency of industrial processes.
Professor Zhu Rong added, Our developed multimodal perception decision system exhibits excellent fine manipulation capabilities and has significant application potential in both home service and industrial manufacturing.
The team is actively engaging with robot manufacturers and automotive companies to advance the industrial application of this technology, aiming to bring it to market sooner rather than later.
This research represents a significant step forward in the field of robotics, offering a glimpse into a future where robots can perform complex tasks with human-like dexterity and intelligence.
Views: 0