Learning by Seeing--Associative Learning of Visual Features Through Mental Simulation of Observed Action

Internal representations employed in cognitive tasks have tobe embodied. The flexible use of such grounded models allowsfor higher-level function like planning ahead, cooperationand communication. But at the same time this flexibilitypresupposes that the utilized internal models are interrelatingmultiple modalities. In this article we present how an internalbody model serving motor control tasks can be recruitedfor learning to recognize movements performed by anotheragent. We show that—as the movements are governed by anequal underlying internal model—it is sufficient to observethe other agent performing a series of movements and thatthere is no supervised learning necessary, i.e. the learningagent does not require access to the performing agents posturalinformation (joint configurations). Instead, through theshared underlying dynamics the mapping can be bootstrappedby the observing agent from the sequence of visual input features.

M. Schilling

Article in conference proceedings