Context-Dependent Recurrent Neural Network Language Modeling
Dr. Geoffrey Zweig
Principal Researcher, Microsoft Research
Recurrent Neural Network Language Models have recently produced some of the best perplexity results on a variety of language modeling tasks. In this talk, we show how to condition the language model on additional, non-textual features. At the simplest level, these may be real-valued topic vectors derived from surrounding text. More generally, the model may be conditioned on representations of a user's interaction history, for example in message dictation or voice search. Similarly, the target language model can be conditioned on source-language information in Machine Translation. After describing the general framework, we present results using LSA and LDA based topic representations, and for some more advanced application areas.
Dr. Geoffrey Zweig is a Principal Researcher at Microsoft Research, where he works on speech recognition technologies, and their applications in Voice Search. At Microsoft, he was one of the core developers of the initial Bing Mobile Voice Search application in 2007, and has since conducted extensive research in the area. More recently, Dr. Zweig spearheaded the development of segmental CRF technology for speech recognition, developed the SCARF toolkit for this, and organized the 2010 JHU summer workshop exploring the topic. Prior to joining Microsoft in 2006, he worked at IBM Research for eight years, most recently as the manager of the Advanced LVSCR research group, responsible for developing English, Arabic and Mandarin speech recognition systems for the DARPA EARS and GALE programs. Dr. Zweig received his PhD in 1998 from the Computer Science Department of the University of California at Berkeley. He has served as Associate Editor of the IEEE-TASLP and currently serves on the IEEE SPS SLTC technical committee. Dr. Zweig is on the editorial board of CSL, and is a senior member of the IEEE and member of the ACL and ACM.