Education

08/2000-12/2004 Purdue University, West Lafayette, IN

Ph.D. in Electrical and Computer Engineering

· Dissertation Title: Structural Event Detection for Rich Transcription of Speech (done at Purdue University and ICSI)

· Advisor: Mary Harper

· Major: Signal Processing and Communication

· Minor: Computer Engineering

09/1997-06/2000 Tsinghua University, Beijing, China

M.S in Electrical Engineering

· Master Thesis Title: Robust Language Model in Speech Recognition

· Major: Information and Signal Processing

09/1992-07/1997 Tsinghua University, Beijing, China

B.S in Electrical Engineering, with honor

· Bachelor Thesis Title: Automatic Segmentation of Chinese Corpus for Speech Recognition

· Major: Information and Signal Processing

Current Funded Activities

12/2004-present International Computer Science Institute, Berkeley, CA

· Parsing speech: investigation of the impact of structural events on parsing speech and the effectiveness of incorporating syntactic knowledge for detecting metadata events. This proposal was written together with Mary Harper and accepted by the NSF/DARPA supported 2005 JHU summer workshop.

Research Experience

12/2004-present International Computer Science Institute, Berkeley, CA

Post-doc Research Fellow

· EARS MDE research, for English and Chinese.

· Dialog Act detection in Meeting corpus.

· Active learning and unsupervised learning for DA modeling and MDE.

07/2002-12/2004 International Computer Science Institute, Berkeley, CA

Research Associate

· In charge of the ICSI/SRI/UW metadata extraction system for the NIST Rich Transcription evaluations. Our system achieved the best overall performance in the two evaluations (RT-03 and RT-04).

· Proposed using the maximum entropy (Maxent) and conditional random fields (CRF) approaches for metadata event detection. Investigated a variety of features (e.g., from POS tagger, chunk parser, and prosody model) in these approaches and feature selection methods.

· Collaborated on generalizing structural event detection algorithms to the multiparty meeting corpus for dialog act segmentation and classification, as well as a multimodal corpus using both a prosody model and a gesture model for sentence boundary detection.

· Investigated machine learning techniques for dealing with the imbalanced data problem in classification.

· Collaborated on using metadata information (sentence boundary hypotheses) for improving speech recognition accuracy by re-segmenting the speech and re- recognition.

· Investigated using voice quality features for word fragment detection and speech synthesis model (fujisaki model) for phrase boundary detection.

08/2000-07/2002 Purdue University, West Lafayette, IN

Research Assistant

· Investigated using a compact graphical representation as the interface between speech recognition and subsequent speech understanding modules.

· Investigated the effectiveness of the combination of different language models for rescoring word lattices or N-best lists.

· Evaluated bottom-up (KL divergence) and top-down (decision tree) approaches for effective triphone acoustic model training.

09/1996-06/2000 Tsinghua University, Beijing, China

Research Assistant

· Investigated different smoothing methods for language model, search algorithms in speech recognition, class-based language model, and language model adaptation.

· Primary project member of the Chinese Government supported speech recognition project. Our team achieved the best system performance in a row of three years.

· Participated in the development of a real-time large vocabulary continuous Chinese speech recognition system, working on lexicon-tree design, Chinese word segmentation algorithm, semantic-based Language Model, multi-pass search algorithm for real-time speech recognition, and command-control module.

10/1999-11/1999 Philips Speech Processing, Vienna, Austria

Speech Recognition System Engineer

· Main developer of a real-time large vocabulary Mandarin continuous speech recognition system, working on lexicon-tree, multi-pass searching algorithm, and language models.

Publications

My Ph.D. thesis “Structural Event Detection for Rich Transcription of Speech” [pdf]

Journals

[1] Yang Liu, Nitesh Chawla, Mary Harper, Elizabeth Shriberg, and Andreas Stolcke, " A Study in Machine Learning from Imbalanced Data for Sentence Boundary Detection in Speech ", To appear in Computer Speech and Language. [pdf]

[2] Yang Liu, Mary Harper, Michael Johnson, and Leah Jamieson, "The Effect of Pruning and Compression on Graphical Representations of the Output of a Speech Recognizer", Computer Speech and Language, Vol. 7, No. 4, pp. 329-356, October 2003. [pdf]

Peer-reviewed Conferences and Workshops

[1] Yang Liu, Mary Harper, Elizabeth Shriberg and Andreas Stolcke, "Using Conditional Random Fields For Sentence Boundary Detection in Speech", To appear in ACL 2005.

[2] Jeremy Ang, Yang Liu, and Elizabeth Shriberg, “Automatic Dialog Act Segmentation and Classification in Multiparty Meetings”, ICASSP 2005. [pdf]

[3] Yang Liu, Elizabeth Shriberg, Andreas Stolcke, Barbara Peskin, Jeremy Ang, Dustin Hillard, Mari Ostendorf, Marcus Tomalin, Phil Woodland, and Mary Harper, “Structural Metadata Research in the EARS Program”, Invited paper. ICASSP 2005. [pdf]

[4] Yang Liu, Andreas Stolcke, Mary Harper, and Elizabeth Shriberg, "Comparing and Combining Generative and Posterior Probability Models: Some Advances in Sentence Boundary Detection in Speech", EMNLP 2004. [pdf]

[5] Yang Liu, Elizabeth Shriberg, Andreas Stolcke, Dustin Hillard, Mari Ostendorf, Barbara Peskin, and Mary Harper, "The ICSI-SRI-UW Metadata Extraction System", ICSLP 2004. [pdf]

[6] Yang Liu, Elizabeth Shriberg, Andreas Stolcke, and Mary Harper, "Using Machine Learning to Cope with Imbalanced Classes in Natural Speech: Evidence from Sentence Boundary and Disfluency Detection", ICSLP 2004. [pdf]

[7] Lei Chen, Yang Liu, Mary Harper, and Elizabeth Shriberg, "Multimodal Model Integration for Sentence Unit Detection", ICMI 2004. [pdf]

[8] Lei Chen, Yang Liu, Eduardo Maia, and Mary Harper, "Evaluating Factors Impacting the Accuracy of Forced Alignments in a Multimodal Corpus", LREC 2004. [pdf]

[9] Dustin Hillard, Mari Ostendorf, Andreas Stolcke, Yang Liu, and Elizabeth Shriberg, "Improving Automatic Sentence Boundary Detection with Confusion Networks", HLT/NAACL 2004. [pdf]

[10] Yang Liu, Elizabeth Shriberg, and Andreas Stolcke, "Automatic Disfluency Identification in Conversational Speech Using Multiple Knowledge Sources", EuroSpeech 2003. [pdf]

[11]Yang Liu, "Word Fragment Identification Using Acoustic-Prosodic Features in Conversational Speech", HLT/NAACL 2003 Student Workshop, 2003. [pdf]

[12]Wen Wang, Yang Liu, and Mary Harper, "Rescoring Effectiveness of Language Models Using Different Levels of Knowledge and Their Integration", ICASSP 2002. [pdf]

[13]Yang Liu, Jiasong Sun, and Zuoying Wang, "Comparisons of Smoothing Methods in Chinese Language Model", ISCSLP 2000. [pdf]

[14]Jiasong Sun, Xiaodong Cui, Zuoying Wang, and Yang Liu, "A Language Model Adaptation Approach Based on Text Classification", ICSLP 2000. [pdf]

Other Workshops

[15]Yang Liu, Elizabeth Shriberg, Andreas Stolcke, Barbara Peskin, and Mary Harper, “The ICSI/SRI/UW RT04 Structural Metadata Extraction System”, EARS RT-04 Workshop, New York, November 2004. [pdf]

[16]Yang Liu, Elizabeth Shriberg, Andreas Stolcke, Barbara Peskin, Jeremy Ang, and Mary Harper, “ICSI-SRI-UW Structural MDE: Modeling, Analysis, and Issues”, EARS RT-04 Workshop presentation, New York, November 2004. [ppt]

[17]Yang Liu, Elizabeth Shriberg, Dustin Hillard et al., "ICSI/SRI/UW RT03F MDE System and Research", EARS RT-03F Workshop presentation, Washington D.C., November 2003. [ppt]

[18]Elizabeth Shriberg, Yang Liu, Mari Ostendorf et al., "Metadata Extraction Research at SRI/ICSI/UW", EARS RT-03S Workshop presentation, Boston, May 2003. [ppt]