Synthetic Data Generation and Vision-based Wrinkle and Keypoint Detection for Bimanual Cloth Manipulation
The paper presents a perception framework for robotic manipulation of textiles, enabling robust visual perception and wrinkle detection.
The paper presents a novel perception framework that integrates a synthetic pipeline and a combination of CNN and YOLOv8-OpenCV for robust textile perception.
Before reading this…
Applications
- →Robotics
- →Textile manipulation
To understand this paper, make sure you know these concepts first:
- Computer visionfind papers →
- Deep learningfind papers →
Abstract
More Like ThisRobotic manipulation of textiles remains challenging because continuous deformation and self-occlusions hinder the robust visual perception required to estimate the cloth's state. To address the lack of annotated real-world data, we developed a Blender-based synthetic pipeline exporting auto-annotated keypoints, and combined manually labeled renders with real-world data to train a wrinkle detector. We present a perception framework integrating a CNN for permutation-invariant keypoint detection and a YOLOv8-OpenCV pipeline to extract grasping points from structural wrinkles. A proposed bimanual algorithm uses this system to stretch fully folded garments via wrinkles, transitioning to keypoint-based ironing once corners emerge. The keypoint model achieves a Mean Position Error (MPE) of 1.7615 pixels. The perception system transfers to physical fabrics without fine-tuning, outperforming baselines that fail in high-occlusion states or yield false positives on severe folds.