Applications of the Soft Bubble Grippers: Visuotactile Sensing for Visuomotor Policy Learning

Contact-rich tasks are challenging for robots to learn, due to the inherently restrictive and rigid constraints imposed by motion planning and hardware. In many tasks, the robot is discouraged from interacting with its environment through touch, either by simply proposing an infeasible trajectory, or by protective stopping upon light contact. As we know from human interactions at a young age, tactile feedback is very important for learning robust and precise manipulation skills. In this work, we examine how tactile feedback can provide auxiliary information for visuomotor policy learning. We do this by replicating and re-designing the Soft Bubble gripper developed by Punyo, a research group at Toyota Research Institute (TRI). The Soft Bubble gripper uses optical sensing to obtain tactile information, also known as a visuotactile sensor. We designed a novel task that is representative of an action that humans commonly rely on tactile feedback to perform. We generate teleoperation demonstration data for training both transformer and diffusion-based policy learning models. From the experimental results, we analyze how task design, hardware design, and architecture/training choices are all contributing factors to performance. We address improvements to be made to original works to leverage visuotactile sensing and what future directions are viable for generalized tactile sensing in robotics.