Perception for the Manipulation of Socks

We consider the perceptual challenges inherent inthe robotic manipulation of previously unseen socks, with theend goal of manipulation by a household robot for laundry. Thetask poses challenging problems in modeling the appearance,shape and configuration of these textile items that tend toexhibit high variability in texture, design, and style while beinghighly articulated objects.

At the heart of our approach is a holistic model of shapeand appearance that facilitates manipulation of those delicateitems—starting even from bunched up instances. We describenovel approaches to two key perceptual problems: (i) Inferringthe configuration of the sock, and (ii) determining which socksshould be paired together.

Robust inference in our model is achieved by strong texturebased classifiers that, alone, are powerful enough to solve problemssuch as inside-out detection. Finally, a reliable predictionof the overall configuration is achieved by combining local cuesin a global model that enforces structural consistency.

We perform an extensive evaluation of different feature typesand classifiers and show strong performance on each subtaskof our approach. Finally, we illustrate our approach with animplementation on the Willow Garage PR2—a general purposerobotic platform.

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Algarve, Portugal, pp. 4877-4884

Vision