Shape-Based Grasping Techniques with Eye Tracking Device

The loss of one hand can significantly affect the level of autonomy and the capability of performing daily living, working and social activities. The current prosthetic solutions contribute in a poor way to overcome these problems due to limitations in the interfaces adopted for controlling the prosthesis and to the lack of force or tactile feedback, thus limiting hand grasping capabilities.

To address this problem, a system was developed to be used with a prosthetic hand. It consisted of an algorithm that allows an adaptive grasping based on the shape of the object. First of all, this system extracted the user’s intention to grab an object through EMG signals (EMG Myo Armband device). In addition, the system was equipped with visual feedback through the use of an Eye Tracking Device and YOLO, a real-time object recognition algorithm. Finally, ROS was used as a framework to gather both EMG signals and visual feedback to guarantee suitable grasping.