6 citations · 6 across the 10 of their papers we have counts for
12 papers
A Tilted Bowl Is Not a Slippery Slope: Compressing Looped Models
Steven Kolawole, Pearse Jim, Opegbemi M. Busoye +2
Looped models reason by applying the same block of weights many times, so compressing that block saves memory traffic on every loop. Compressed looped models, however, often collap…
Sample What You Say: Aligning Language Models to Sample the Distributions They State
Kasra Arabi, Virginia Smith, Chhavi Yadav
Language models are increasingly used to sample from a specified distribution, for instance, to simulate survey respondents or generate synthetic data. Instruction-tuned models can…
EpiKV: Epiphany-Aware KV Cache Eviction Without the Attention Matrix
Steven Kolawole, Virginia Smith
Reasoning models can generate chains of thought tens of thousands of tokens long, making the key--value (KV) cache that holds them a major bottleneck for inference throughput. Exis…
What Survives When You Compress a Recursive Reasoner for the Edge?
Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye +2
Recursive reasoning models solve hard structured tasks with a few million parameters by iterating a latent state, but deploying them on edge hardware means compressing them -- and…
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
Kevin Kuo, Chhavi Yadav, Virginia Smith
Recent defenses for safeguarding open-weight large language models (LLMs) are intended to prevent adversarial usage. Underlying these defenses is an assumption that new harmful beh…
Curriculum Learning for Safety Alignment
Sandeep Kumar, Virginia Smith, Chhavi Yadav
Direct Preference Optimisation (DPO) is widely used for safety alignment in large language models. However, prior work shows it is brittle and exhibits poor out-of-distribution (OO…
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3
Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater representational plasticity, Low-Rank Adaptati…
Automated Concept Discovery for LLM-as-a-Judge Preference Analysis
James Wedgwood, Chhavi Yadav, Virginia Smith
Large Language Models (LLMs) are increasingly used as scalable evaluators of model outputs, but their preference judgments exhibit systematic biases and can diverge from human eval…
Semantic Agreement Enables Efficient Open-Ended LLM Cascades
Duncan Soiffer, Steven Kolawole, Virginia Smith
Cascade systems route computational requests to smaller models when possible and defer to larger models only when necessary, offering a promising approach to balance cost and quali…
PARALLELPROMPT: Extracting Parallelism from Large Language Model Queries
Steven Kolawole, Keshav Santhanam, Virginia Smith +1
LLM serving systems typically treat user prompts as monolithic inputs, optimizing inference through decoding tricks or inter-query batching. However, many real-world prompts contai…
Agreement-Based Cascading for Efficient Inference
Steven Kolawole, Don Dennis, Ameet Talwalkar +1
Adaptive inference schemes reduce the cost of machine learning inference by assigning smaller models to easier examples, attempting to avoid invocation of larger models when possib…
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
Steven Kolawole, Lucio Dery, Jean-François Kagy +3
Structured pruning is a promising approach to create smaller, faster large language models. However, existing methods typically rely on computing the gradient via backward passes,…