3 citations · 3 across the 10 of their papers we have counts for
4 papers · 1 filter
SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation
Zachary Morrow, Michael Penwarden, Brian Chen +3
Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are flexible function approximators, but their accuracy can require substantial network depth, width, and trainabl…
PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
Haonan Wang, Brian Chen, Siquan Li +4
Parameter-Efficient Fine-Tuning (PEFT) methods have become crucial for rapidly adapting large language models (LLMs) to downstream tasks. Prefix-Tuning, an early and effective PEFT…
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
Yezhen Wang, Zhouhao Yang, Brian K Chen +4
Building upon the success of low-rank adapter (LoRA), low-rank gradient projection (LoRP) has emerged as a promising solution for memory-efficient fine-tuning. However, existing Lo…
Auditing language models for hidden objectives
Samuel Marks, Johannes Treutlein, Trenton Bricken +32
We study the feasibility of conducting alignment audits: investigations into whether models have undesired objectives. As a testbed, we train a language model with a hidden objecti…