Learning to Detect Visual Grasp Affordance

TitleLearning to Detect Visual Grasp Affordance
Publication TypeJournal Article
Year of Publication2015
AuthorsSong HOh, Fritz M, Göhring D, Darrell T
Other Numbers3747

Appearance-based estimation of grasp affordances is desirable when 3-D scans become unreliable due to clutter or material properties. We develop a general framework for estimating grasp affordances from 2-D sources, including local texture-like measures as well as object-category measures that capture previously learned grasp strategies. Local approaches to estimating grasp positions have been shown to be effective in real-world scenarios, but are unable to impart object-level biases and can be prone to false positives. We describe how global cues can be used to compute continuous pose estimates and corresponding grasp point locations, using a max-margin optimization for category-level continuous pose regression. We provide a novel dataset to evaluate visual grasp affordance estimation; on this dataset we show that a fused method outperforms either local or global methods alone, and that continuous pose estimation improves over discrete output models. Finally, we demonstrate our autonomous object detection and grasping system on the Willow Garage PR2 robot.

Bibliographic Notes

To appear in the IEEE Transactions on Automation Science and Engineering

Abbreviated Authors

H. O. Song, M. Fritz, D. Goehring, and T. Darrell

ICSI Research Group


ICSI Publication Type

Article in journal or magazine