Abstract:  

This talk reviews my early work at ATR on the use of prosody in speech synthesis and continues to outline my present and future work at Trinity College Dublin on multimodal interaction ... it started with synthesis in a speech translation environment but developed into dialogue systems and hmi ... involving a lot of corpora on the way ... expect little stats and few formulae but lots of insights into how speech works and what types of new technology we can make for speech processing in conversational interaction. The talk also describes recent work on the collection, analysis, and modelling of multimodal conversational speech data and shows that many features of expressive speech and bodily gesture serve both to control and display inter-personal and social aspects of an interaction rather than function primarily as linguistic elements in the discourse.  The talk will show how BOTH linguistic and paralinguistic aspects of communication are combined in a natural way so that the social aspects of interpersonal communication function smoothly alongside the linguistic aspects of social interaction.