Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention

Sounak Mondal, Dimitris Samaras, Gregory Zelinsky +1

We introduce a novel learning problem: decoding gaze into natural language descriptions of human goals across diverse visual tasks. Unlike prior work, which frames gaze decoding as…