Timely Object Recognition

TitleTimely Object Recognition
Publication TypeMiscellaneous
Year of Publication2012
AuthorsKarayev, S., Baumgartner T., Fritz M., & Darrell T.
Other Numbers3356

In a large visual multi-class detection framework, the timeliness of results can be crucial. Our method for timely multi-class detection aims to give the best possible performance at any single point after a start time; it is terminated at a deadline time. Toward this goal, we formulate a dynamic, closed-loop policy that infers the contents of the image in order to decide which detector to deploy next. In contrast to previous work, our method significantly diverges from the predominant greedy strategies, and is able to learn to take actions with deferred values. We evaluate our method with a novel timeliness measure, computed as the area under an Average Precision vs. Time curve. Experiments are conducted on the eminent PASCAL VOC object detection dataset. If execution is stopped when only half the detectors have been run, our method obtains 66% better AP than a random ordering, and 14% better performance than an intelligent baseline. On the timeliness measure, our method obtains at least 11% better performance. Our code, to be made available upon publication, is easily extensible as it treats detectors and classifiers as black boxes and learns from execution traces using reinforcement learning.


This research was made with Government support under and awarded by DoD, Air Force Office ofScientific Research, National Defense Science and Engineering Graduate (NDSEG) Fellowship, 32CFR 168a.

Bibliographic Notes

Presented at the 26th Annual Conference on Neural Information Processing Systems (NIPS 2012), Lake Tahoe, Nevada

Abbreviated Authors

S. Karayev, T. Baumgartner, M. Fritz, and T. Darrell

ICSI Research Group


ICSI Publication Type

Talk or presentation