Training a Multilingual Sportscaster: Using Perceptual Context to Learn Language

David L. Chen, Joohyun Kim, Raymond J. Mooney

We present a novel framework for learning to interpret and generate language using only perceptual context as supervision. We demonstrate its capabilities by developing a system th…