10 papers
Adam's Law: Textual Frequency Law on Large Language Models
Hongyuan Adam Lu, Z. L., Victor Wei +5
While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel re…
Future Validity is the Missing Statistic: From Impossibility to -Estimation for Grammar-Faithful Speculative Decoding
Wenhua Nie, Zijie Meng, Kun Zou +5
Grammar-constrained generation is often combined with local vocabulary masking and speculative decoding, but the resulting sampling law is not the grammar-conditional distribution…
Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works
Wenhua Nie, Jianan Wu, Junlin Liu +6
Group Relative Policy Optimization (GRPO) is a standard algorithm for reinforcement learning from verifiable rewards, but its group-mean-centered advantage can fail under binary re…
Phonemes vs. Projectors: An Investigation of Speech-Language Interfaces for LLM-based ASR
Ziwei Li, Lukuang Dong, Saierdaer Yusuyin +2
Integrating pretrained speech encoders with large language models (LLMs) is promising for ASR, but performance and data efficiency depend on the speech-language interface. A common…
SLaB: Sparse-Lowrank-Binary Decomposition for Efficient Large Language Models
Ziwei Li, Yuang Ma, Yi Kang
The rapid growth of large language models (LLMs) presents significant deployment challenges due to their massive computational and memory demands. While model compression, such as…
STEAR: Layer-Aware Spatiotemporal Evidence Intervention for Hallucination Mitigation in Video Large Language Models
Linfeng Fan, Yuan Tian, Ziwei Li +1
Video Large Language Models (Video-LLMs) remain prone to spatiotemporal hallucinations, often generating visually unsupported details or incorrect temporal relations. Existing miti…