Publication Details
Title: The Tao of ATWV: Probing the Mysteries of Keyword Search Performance
Author: S. Wegmann, A. Faria, A. Janin, K. Riedhammer, and N. Morgan
Bibliographic Information: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU 2013), Olomouc, Czech Republic
Date: December 2013
Research Area: Speech
Type: Article in conference proceedings
PDF: https://www.icsi.berkeley.edu/pubs/speech/taoatwv13.pdf
Overview:
In this paper we apply diagnostic analysis to gain a deeper understanding of the performance of the the keyword search system that we have developed for conversational telephone speech in the IARPA Babel program. We summarize the Babel task, its primary performance metric, ``actual term weighted value'' (ATWV), and our recognition and keyword search systems. Our analysis uses two new oracle ATWV measures, a bootstrap-based ATWV confidence interval, and includes a study of the underpinnings of the large ATWV gains due to system combination. This analysis quantifies the potential ATWV gains from improving the number of true hits and the overall quality of the detection scores in our system's posting lists. It also shows that system combination improves our systems' ATWV via a small increase in the number of true hits in the posting lists.
Acknowledgements:
This work was supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Defense US Army Research Laboratory contract number W911NF-12-C-0014 ("Spoken WOrdsearch with Rapid Development and Frugal Invariant Subword Hierarchies - Swordfish"). The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoD/ARL, or the U.S. Government
Bibliographic Reference:
S. Wegmann, A. Faria, A. Janin, K. Riedhammer, and N. Morgan. The Tao of ATWV: Probing the Mysteries of Keyword Search Performance. Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU 2013), Olomouc, Czech Republic, December 2013
Author: S. Wegmann, A. Faria, A. Janin, K. Riedhammer, and N. Morgan
Bibliographic Information: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU 2013), Olomouc, Czech Republic
Date: December 2013
Research Area: Speech
Type: Article in conference proceedings
PDF: https://www.icsi.berkeley.edu/pubs/speech/taoatwv13.pdf
Overview:
In this paper we apply diagnostic analysis to gain a deeper understanding of the performance of the the keyword search system that we have developed for conversational telephone speech in the IARPA Babel program. We summarize the Babel task, its primary performance metric, ``actual term weighted value'' (ATWV), and our recognition and keyword search systems. Our analysis uses two new oracle ATWV measures, a bootstrap-based ATWV confidence interval, and includes a study of the underpinnings of the large ATWV gains due to system combination. This analysis quantifies the potential ATWV gains from improving the number of true hits and the overall quality of the detection scores in our system's posting lists. It also shows that system combination improves our systems' ATWV via a small increase in the number of true hits in the posting lists.
Acknowledgements:
This work was supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Defense US Army Research Laboratory contract number W911NF-12-C-0014 ("Spoken WOrdsearch with Rapid Development and Frugal Invariant Subword Hierarchies - Swordfish"). The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoD/ARL, or the U.S. Government
Bibliographic Reference:
S. Wegmann, A. Faria, A. Janin, K. Riedhammer, and N. Morgan. The Tao of ATWV: Probing the Mysteries of Keyword Search Performance. Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU 2013), Olomouc, Czech Republic, December 2013