23 papers
Defense-in-Depth for LLMs: Evaluating Memory Gates Against Activation-Induced and Memory-Induced Sycophancy
Ritvij Sharma, Russell Dlugosz, Ryan Zhou +1
Long-term memory allows Large Language Models (LLMs) to maintain personalized context across interactions, but retrieved user history can induce memory-induced sycophancy, causing…
Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition
Devesh Tiwari, Camille Davis, Shivank Sinha +3
Linear probes can decode safety-relevant concepts such as truthfulness from language-model activations, but probe accuracy may show only decodability, not that the features the pro…
Evaluation Awareness Shifts from Format to Context with Model Scale
Navraj Singh, Maheep Chaudhary
Evaluation awareness poses an unprecedented threat to model evaluation, but the mechanisms by which models detect it remain unknown. This study focuses on determining this and iden…
Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition
Aarnav Choudhary, Matheus Fonseca Rocha, Jiwon Seo +2
Model merging is often used to combine capabilities from separately fine-tuned models without additional training, but it is unclear whether standard merging methods preserve multi…
Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Models
Jasmine Brazilek, Joel Christoph, Maheep Chaudhary +4
Previous research has evaluated animal welfare using question-and-answer benchmarks. This study investigates whether these evaluations also hold in agentic settings. The agents may…
Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
Jasmine Brazilek, Maheep Chaudhary, Zoe Lu +1
Multi-agent systems routinely place one AI agent in authority over another. When a subordinate refuses a task, the manager chooses the outcome: it can renegotiate, report the failu…
Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation
Uwe König, Hamza Kazmi, Ruizhe Li +1
Distillation of a language model intended to transfer benign behavior to a student model may also transfer undesirable characteristics, if they are present in the teacher model, a…
LLM Scheming Inversely Scales with Pretraining Language Coverage
Nathan Truong, Aryan Panda, Rayming Ye +2
With the growing capabilities of frontier models, AI alignment becomes increasingly critical in high-risk deployment settings. While recent work has empirically demonstrated in-con…
Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
Jeanmely Rojas Nunez, Viraj Sawant, Nathan Allen +4
Fine-tuning large language models (LLMs) frequently induces catastrophic forgetting of prior capabilities. Recent work has shown that reinforcement learning (RL) retains prior capa…
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
Ishaan Kelkar, Nebras Alam, Vikram Kakaria +3
Language models are often sycophantic: they agree with a user's stated opinion whether or not it is correct. Prior work has shown that this trait can be controlled by steering a mo…
Why Limit the Residual Stream to Layers and Not Tokens? Persistent Memory for Continuous Latent Reasoning
Mujtaba Farhan, Maheep Chaudhary
Large language models (LLMs) have demonstrated remarkable reasoning abilities on mathematical and multi-hop planning tasks. The CoCoNuT (Chain of Continuous Thought) paradigm~\cite…
In-Context Environments Induce Evaluation-Awareness in Language Models
Maheep Chaudhary
Humans often become more self-aware under threat, yet can lose self-awareness when absorbed in a task; we hypothesize that language models exhibit environment-dependent \textit{eva…
MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav +4
Defending LLMs against adversarial jailbreak attacks remains an open challenge. Existing defenses rely on binary classifiers that fail when adversarial input falls outside the lear…
Broken Chains: The Cost of Incomplete Reasoning in LLMs
Ian Su, Gaurav Purushothaman, Jey Narayan +5
Reasoning-specialized models like OpenAI's 5.1 and DeepSeek-V3.2 allocate substantial inference compute to extended chain-of-thought (CoT) traces, yet reasoning tokens incur signif…
Weight space Detection of Backdoors in LoRA Adapters
David Puertolas Merenciano, Ekaterina Vasyagina, Kevin Zhu +2
LoRA adapters let users fine-tune large language models (LLMs) efficiently. However, LoRA adapters are shared through open repositories like Hugging Face Hub \citep{huggingface_hub…
SALT: Steering Activations towards Leakage-free Thinking in Chain of Thought
Shourya Batra, Pierce Tillman, Samarth Gaggar +6
As Large Language Models (LLMs) evolve into personal assistants with access to sensitive user data, they face a critical privacy challenge: while prior work has addressed output-le…
Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits
Dev Patel, Gabrielle Gervacio, Diekola Raimi +5
Large Language Models require substantial computational resources for inference, posing deployment challenges. While dynamic pruning offers superior efficiency over static methods…
Optimizing Chain-of-Thought Confidence via Topological and Dirichlet Risk Analysis
Abhishek More, Anthony Zhang, Nicole Bonilla +4
Chain-of-thought (CoT) prompting enables Large Language Models to solve complex problems, but deploying these models safely requires reliable confidence estimates, a capability whe…
PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Cases
Sri Vatsa Vuddanti, Aarav Shah, Satwik Kumar Chittiprolu +4
Tool-augmented language agents frequently fail in real-world deployment due to tool malfunctions--timeouts, API exceptions, or inconsistent outputs--triggering cascading reasoning…
LIFT: Likelihood-Based Intervention Faithfulness Training
Anand Swaroop, Akshat Nallani, Saksham Uboweja +7
Chain-of-thought (CoT) reasoning improves LLM accuracy on complex tasks, but CoTs are often unfaithful; that is, the model does not rely on the stated reasoning steps to produce it…
Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimization
Nathan Egbuna, Saatvik Gaur, Sunishchal Dev +2
Test-time optimization remains impractical at scale due to prohibitive inference costs--techniques like iterative refinement and multi-step verification can require …
Evaluation Awareness Scales Predictably in Open-Weights Large Language Models
Maheep Chaudhary, Ian Su, Nikhil Hooda +6
Large language models (LLMs) can internally distinguish between evaluation and deployment contexts, a behaviour known as \emph{evaluation awareness}. This undermines AI safety eval…
Hydra: A Modular Architecture for Efficient Long-Context Reasoning
Siddharth Chaudhary, Dev Patel, Maheep Chaudhary +1
The quadratic complexity of transformers fundamentally limits reasoning system deployment in resource-constrained and long-context settings. We introduce Hydra, a modular architect…