5 papers
DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation
Vasileios Baltatzis, Mert Inan, Connor Gillis +4
Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce D…
Bootstrapping Sign Language Annotations with Sign Language Models
Colin Lea, Vasileios Baltatzis, Connor Gillis +3
AI-driven sign language interpretation is limited by a lack of high-quality annotated data. New datasets including ASL STEM Wiki and FLEURS-ASL contain professional interpreters an…
How Pragmatics Shape Articulation: A Computational Case Study in STEM ASL Discourse
Saki Imai, Lee Kezar, Laurel Aichler +5
Most state-of-the-art sign language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, hum…
Towards AI-driven Sign Language Generation with Non-manual Markers
Han Zhang, Rotem Shalev-Arkushin, Vasileios Baltatzis +7
Sign languages are essential for the Deaf and Hard-of-Hearing (DHH) community. Sign language generation systems have the potential to support communication by translating from writ…
Generating Signed Language Instructions in Large-Scale Dialogue Systems
Mert İnan, Katherine Atwell, Anthony Sicilia +2
We introduce a goal-oriented conversational AI system enhanced with American Sign Language (ASL) instructions, presenting the first implementation of such a system on a worldwide m…