Prosodic and temporal features for language modeling for dialog
- If we can model the cognitive and communicative processes underlying speech, we should be able to better predict what a speaker will do. With this idea as inspiration, we examine a number of prosodic and timing features as potential sources of information on what words the speaker is likely to say next. In spontaneous dialog we find that word probabilities do vary with such features. Using perplexity as the metric, the most informative of these included recent speaking rate, volume, and pitch, and time until end of utterance. Using simple combinations of such features to augment trigram language models gave up to a 8.4% perplexity benefit on the Switchboard corpus, and up to a 1.0% relative reduction in word error rate (0.3% absolute) on the Verbmobil II corpus.
| Author details: | Nigel G. Ward, Alejandro Vega, Timo Baumann |
|---|---|
| DOI: | https://doi.org/10.1016/j.specom.2011.07.009 |
| ISSN: | 0167-6393 |
| Title of parent work (English): | Speech communication |
| Publisher: | Elsevier |
| Place of publishing: | Amsterdam |
| Publication type: | Article |
| Language: | English |
| Year of first publication: | 2012 |
| Publication year: | 2012 |
| Release date: | 2017/03/26 |
| Tag: | Dialog dynamics; Dialog state; Interlocutor behavior; Perplexity; Prediction; Prosody; Speech recognition; Switchboard corpus; Verbmobil corpus; Word probabilities |
| Volume: | 54 |
| Issue: | 2 |
| Number of pages: | 14 |
| First page: | 161 |
| Last Page: | 174 |
| Funding institution: | NSF [IIS-0415150, IIS-0914868]; US Army Research, Development and Engineering Command; USC Institute for Creative Technologies; DFG |
| Organizational units: | Humanwissenschaftliche Fakultät / Strukturbereich Kognitionswissenschaften / Department Linguistik |
| Peer review: | Referiert |
| Institution name at the time of the publication: | Humanwissenschaftliche Fakultät / Institut für Linguistik / Allgemeine Sprachwissenschaft |
