Commanding Grasping Robot through Virtual Reality and Simulated Wrenches
In recent years, the demand for object manipulation in hostile environments, using natural human body movements, has driven the development of various teleoperation interfaces. However, traditional teleoperation, where the robot and environment are viewed through a camera, often results in non-ergonomic user postures and occlusion issues.
​
This work proposes a novel teleoperation method that integrates grasping theory with virtual reality. To address occlusion challenges, an experimental teleoperation setup has been designed, enabling continuous interaction between the human hand, a virtual kinematic hand model, and the robotic manipulator. The system features an interactive virtual grasping interface, where data from the user’s hand movements are transmitted to the manipulator, allowing it to replicate the grasping actions in the physical world. The user experiences force feedback from the virtual environment via wearable haptic devices.
​
Future developments aim to enhance user experience by masking the physical robot, enabling the user to identify with a virtual hand in the real environment, reducing occlusion and mitigating ergonomic challenges.
Virtual environment.
Grasping virtual objects is one of the most challenging tasks in Virtual Reality, due to factors such as object shape and discrepancies in physical behavior. As a hypothesis, this work proposes the use of a sphere, matching the object's mass, which encloses the target object by contacting its outermost points.
OptiTrack Motive system to track the user's hand.
The tracking system employs infrared (IR) light emitted by the cameras, which is reflected by the passive markers and detected by the camera sensors. This enables precise determination of the markers' positions.
Manipulation task.
To track the user's hand, the system utilizes the Leap Motion Controller for palm positioning, an Inertial Measurement Unit (IMU) for palm orientation, and the CyberGlove III to capture the joint angles of the fingers.