4 citations · 4 across the 10 of their papers we have counts for
10 papers
Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference
Zixuan Lan, Yanhong Li, Jiawei Zhou
Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix…
Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
Xinghong Fu, Yanhong Li, Georgios Papaioannou +1
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has be…
Distilling to Hybrid Attention Models via KL-Guided Layer Selection
Yanhong Li, Songlin Yang, Shawn Tan +4
Distilling pretrained softmax attention Transformers into more efficient hybrid architectures that interleave softmax and linear attention layers is a promising approach for improv…
OKBench: Democratizing LLM Evaluation with Fully Automated, On-Demand, Open Knowledge Benchmarking
Yanhong Li, Tianyang Xu, Kenan Tang +3
Knowledge-intensive question answering is central to large language models (LLMs) and is typically assessed using static benchmarks derived from sources like Wikipedia and textbook…
Text or Pixels? It Takes Half: On the Token Efficiency of Visual Text Inputs in Multimodal LLMs
Yanhong Li, Zixuan Lan, Jiawei Zhou
Large language models (LLMs) and their multimodal variants can now process visual inputs, including images of text. This raises an intriguing question: can we compress textual inpu…
On the Predictive Power of Representation Dispersion in Language Models
Yanhong Li, Ming Li, Karen Livescu +1
We show that a language model's ability to predict text is tightly linked to the breadth of its embedding space: models that spread their contextual representations more widely ten…
Context-Efficient Retrieval with Factual Decomposition
Yanhong Li, David Yunis, David McAllester +1
There has recently been considerable interest in incorporating information retrieval into large language models (LLMs). Retrieval from a dynamically expanding external corpus of te…
Chunk-Distilled Language Modeling
Yanhong Li, Karen Livescu, Jiawei Zhou
We introduce Chunk-Distilled Language Modeling (CD-LM), an approach to text generation that addresses two challenges in current large language models (LLMs): the inefficiency of to…
When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
Yanhong Li, Chenghao Yang, Allyson Ettinger
Recent studies suggest that self-reflective prompting can significantly enhance the reasoning capabilities of Large Language Models (LLMs). However, the use of external feedback as…
Self-Supervised Video Transformers for Isolated Sign Language Recognition
Marcelo Sandoval-Castaneda, Yanhong Li, Diane Brentari +2
This paper presents an in-depth analysis of various self-supervision methods for isolated sign language recognition (ISLR). We consider four recently introduced transformer-based a…