Using Robotic Exploratory Procedures to Learn the Meaning of Haptic Adjectives

TitleUsing Robotic Exploratory Procedures to Learn the Meaning of Haptic Adjectives
Publication TypeConference Paper
Year of Publication2013
AuthorsChu, V., McMahon I., Riano L., McDonald C. G., He Q., Perez-Tejada J. Martinez, Arrigo M., Fitter N., Nappo J. C., Darrell T., & Kuchenbecker K. J.
Other Numbers3447

Delivering on the promise of real-world robotics will require robots that can communicate with humans through natural language by learning new words and concepts through their daily experiences. Our research strives to create a robot that can learn the meaning of haptic adjectives by directly touching objects. By equipping the PR2 humanoid robot with state-of-the-art biomimetic tactile sensors that measure temperature, pressure, and fingertip deformations, we created a platform uniquely capable of feeling the physical properties of everyday objects. The robot used five exploratory procedures to touch 51 objects that were annotated by human participants with 34 binary adjective labels. We present both static and dynamic learning methods to discover the meaning of these adjectives from the labeled objects, achieving average F1 scores of 0.57 and 0.79 on a set of eight previously unfelt items.


This material is based upon work supported by the DefenseAdvanced Research Projects Agency (DARPA) in the UnitedStates as part of Activity E within the Broad OperationalLanguage Translation (BOLT) program.

Bibliographic Notes

Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), Karlsruhe, Germany

Abbreviated Authors

V. Chu, I. McMahon, L. Riano, C. G. McDonald, Q. He, J. Martinez Perez-Tejada, M. Arrigo, N. Fitter, J. C. Nappo, T. Darrell, and K. J. Kuchenbecker

ICSI Research Group


ICSI Publication Type

Article in conference proceedings