Flexible Piezoelectric Tactile Sensing System for Intelligent Object Recognition

How can a robot recognize an object within hundreds of milliseconds while grasping it?

Object recognition is a fundamental skill for robotic manipulation, enabling robots to adapt their grasping strategies based on the properties of the objects being handled. Although vision-based systems are widely used, they often struggle with transparent objects and cannot perceive physical properties such as stiffness, hardness, and texture.

Artificial tactile sensing offers a complementary solution by providing information obtained through direct physical interaction between the robotic gripper and the object. However, a challenging question remains: can a robot reliably recognize an object in the earliest stages of grasping while adhering to the strict timing constraints of real-time manipulation?

In an attempt to answer this question, the researchers developed an electronic skin that uses flexible piezoelectric sensors for early-stage intelligent object recognition. This system integrates soft-sensing arrays with multichannel embedded electronics, enabling rapid acquisition of tactile signals during grasping. Two flexible arrays, each composed of 16 sensors, are designed as soft, customizable sensing caps that can be mounted on robotic end-effectors. This setup enables the gripper to detect contact information from the very first moments of interaction with an object.

To evaluate the effectiveness of the system, the sensing caps were mounted on the two grippers of the TIAGo robot, and a representative dataset was collected during the grasping of six everyday objects. The tactile signals obtained were used to develop and compare various machine learning and deep learning approaches. These approaches included feature-based classification using Support Vector Machines; raw-signal learning with 1D convolutional neural networks and convolutional recurrent neural networks; and a spatiotemporal heatmap representation generated from raw signals processed with 2D convolutional neural networks.

A systematic analysis was performed to assess how the number of sensors and the length of the temporal window influence both classification accuracy and real-time feasibility. The results confirmed that increasing the number of sensing elements enhances the performance of all evaluated approaches, highlighting the importance of a distributed sensing system. The highest accuracy was achieved by using the first 100 milliseconds of tactile data after contact onset, demonstrating that reliable object recognition can already be performed during the initial stages of grasping.

Among the evaluated approaches, the proposed spatiotemporal heatmap representation, combined with a 2D convolutional neural network, achieved the best results, achieving an F1-score of 0.9903 when all 32 sensing elements were used. The optimal configuration was then deployed on an embedded platform with constraints, enabling end-to-end recognition in approximately 101 milliseconds from the onset of contact, while consuming only 63.8 microjoules (μJ) per inference. These results demonstrate the effectiveness of the proposed distributed flexible tactile sensing system for fast, real-time, and early-stage object recognition, highlighting its potential for robotic manipulation.