Robot Textile and Fabric Inspection and Manipulation

Through the course of the project we will solve these problems. We will 1) develop a vision-based system that allows segmentation, characterization and inspection of the manipulated textiles/fabrics. It will be based on robust deep-learning-based multimodal segmentation and detection of key points relevant for grasping, as well as on unsupervised learning for defect detection. 2) We will develop and demonstrate effective goal directed handling and manipulation of textile/fabric objects. This will be achieved through learning of appropriate motion policies, where advanced deep learning and reinforcement learning methods in simulation and in the real world, sim-to-real methods and training of new, end-to-end vision-to-motion deep neural networks will be applied. 3) Finally, we will develop means to plan a sequence of actions based on a novel state representation of textile. It will be used to form graphs of states where each edge is a transition with information on the needed robot action.

To demonstrate the technological advances, we will implement a bimanual robot cell for textile and fabric logistics at TRL4. It will detect, flatten, inspect and fold textiles and fabrics into desired goal states. The presented demonstration will cover all the major aspects of the project in perception/inspection and handling/manipulation of such deformable objects.