4 papers
When Can Attention Heads Be Statically Defined?
Weixian Waylon Li, Yintao Tai, Marcio Fonseca +1
Some attention heads learn similar patterns across inputs. Reusing these patterns could reduce training cost by avoiding repeated query-key score computation and softmax. Through c…
MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts
Chen Hu, Yintao Tai, Antonio Vergari +2
Pixel-based language models are gaining momentum as alternatives to traditional token-based approaches, promising to circumvent tokenization challenges; however, existing models re…
PIXAR: Auto-Regressive Language Modeling in Pixel Space
Yintao Tai, Xiyang Liao, Alessandro Suglia +1
Recent work showed the possibility of building open-vocabulary large language models (LLMs) that directly operate on pixel representations. These models are implemented as autoenco…
Learning to Simulate Natural Language Feedback for Interactive Semantic Parsing
Hao Yan, Saurabh Srivastava, Yintao Tai +3
Interactive semantic parsing based on natural language (NL) feedback, where users provide feedback to correct the parser mistakes, has emerged as a more practical scenario than the…