5 citations · 6 across the 27 of their papers we have counts for
6 papers · 1 filter
When Should LLMs Reason? Adaptive Inference Routing via Entropy Dynamics
Wei Xia, Haoqing Wang, Zhi-Hong Deng +1
Chain-of-thought (CoT) reasoning can improve LLM performance, but its benefits vary across tasks, models, and individual queries. On some tasks, CoT provides marginal or negative a…
Priming: Hybrid State Space Models From Pre-trained Transformers
Aditya Chattopadhyay, Elvis Nunez, Prannay Kaul +6
Hybrid State-Space models combine Attention with recurrent State-Space Model (SSM) layers, balancing eidetic memory from Attention with compressed fading memory from SSMs. This yie…
Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
Xiaoze Liu, Dhananjay Ram, Yuting Zhang +3
We introduce Mutual Reinforcement Learning, a framework for concurrent RL post-training in which heterogeneous LLM policies exchange typed experience while keeping separate paramet…
Reinforcement-aware Knowledge Distillation for LLM Reasoning
Zhaoyang Zhang, Shuli Jiang, Yantao Shen +6
Reinforcement learning (RL) post-training has recently driven major gains in long chain-of-thought reasoning large language models (LLMs), but the high inference cost of such model…
Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression
Liangzu Peng, Aditya Chattopadhyay, Luca Zancato +3
Linear State-Space Models (SSMs) offer an efficient alternative to softmax Attention with constant memory and linear compute, but their lossy, fading summary of the past hurts reca…
Beyond Hidden-Layer Manipulation: Semantically-Aware Logit Interventions for Debiasing LLMs
Wei Xia
We proposed Static and Dynamic -- two zero-shot logits-layer debiasing methods. Dynamic reduces bias by up to 70% with minimal fluency loss. Logits intervention outperforms hidden-…