6 papers · 1 filter
FlashMap: Flow Map Speculative Decoding
Jeff Shen, Zhejian Zhou, Ryan Lee +2
Speculative decoding accelerates large language model (LLM) inference by using a lightweight drafter to propose a block of tokens that the target model verifies in one parallel pas…
How To Train Your World Model: Fine-tuning vs RAG for LM-based World Modeling
Dhananjay Ashok, Shantanu Agarwal, Vivek Datla +2
World models (WMs) simulate the transition dynamics of environments, enabling agents to plan over the consequences of their actions. In text-based environments, fine-tuning a Langu…
Toward Interactive Understanding of Code APIs
Dhananjay Ashok, Jesse Thomason, Jonathan May
Empowered by advances in Language Model agents, systems have made substantial strides in code generation and understanding. However, these approaches often rely on read access to t…
Sparse Layers are Critical to Scaling Looped Language Models
Ryan Lee, Jacob Biloki, Edward J. Hu +1
Looped language models repeat a set of transformer layers through depth, reducing memory costs and providing natural early-exit points at loop boundaries. However, looped models do…
How Language Models Process Negation
Zhejian Zhou, Tianyi Zhou, Robin Jia +1
We study how Large Language Models (LLMs) process negation mechanistically. First, we establish that even though open-weight models often provide wrong answers to questions involvi…
Conceptual Steganography
Zhejian Zhou, Jonathan May
Language Models (LMs) emit Chains-of-Thought (CoTs) that drive much of their capability. However, the same sequence that carries useful reasoning can also covertly convey messages:…