3 citations · 3 across the 10 of their papers we have counts for
10 papers
SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models
Oshayer Siddique, J. M Areeb Uzair Alam, Md Jobayer Rahman Rafy +3
Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning. Sparse autoencoders (SAEs) can make such inter…
StemFX: Learning Mixing Style Representations via Autoregressive FX Chain Prediction on Source-Separated Stems
Yuan-Chiao Cheng, Jui-Te Wu, Brian Chen +3
Audio mixing style encompasses the artistic and technical decisions a mix engineer makes, including level balancing, spatialization, and the choice, ordering, and parameterization…
Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models
Brian K Chen, Chong Wu, Kenji Kawaguchi
Diffusion language models (DLMs) can revise tokens bidirectionally, but standard decoding procedures often adapt them to left-to-right generation by producing text block by block.…
Logical Judgments Under Pressure: Diagnosing Syllogistic Stability with Learned Soft Prefixes
Brian K Chen
To test how correct logical judgments respond to learned context, we prepend a soft prefix to an exactly labeled syllogistic reasoning benchmark while keeping the model fixed. Soft…
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
Adly Templeton, Tom Conerly, Jonathan Marcus +23
We demonstrate that sparse autoencoders can extract interpretable features from Claude 3 Sonnet, a production-scale language model, addressing the open question of whether dictiona…
SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation
Zachary Morrow, Michael Penwarden, Brian Chen +3
Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are flexible function approximators, but their accuracy can require substantial network depth, width, and trainabl…
PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
Haonan Wang, Brian Chen, Siquan Li +4
Parameter-Efficient Fine-Tuning (PEFT) methods have become crucial for rapidly adapting large language models (LLMs) to downstream tasks. Prefix-Tuning, an early and effective PEFT…
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
Yezhen Wang, Zhouhao Yang, Brian K Chen +4
Building upon the success of low-rank adapter (LoRA), low-rank gradient projection (LoRP) has emerged as a promising solution for memory-efficient fine-tuning. However, existing Lo…
Auditing language models for hidden objectives
Samuel Marks, Johannes Treutlein, Trenton Bricken +32
We study the feasibility of conducting alignment audits: investigations into whether models have undesired objectives. As a testbed, we train a language model with a hidden objecti…
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian K Chen, Tianyang Hu, Hui Jin +2
In-Context Learning (ICL) has been a powerful emergent property of large language models that has attracted increasing attention in recent years. In contrast to regular gradient-ba…