8 citations · 18 across the 15 of their papers we have counts for
3 papers · 1 filter
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Johannes Knittel, Alec Helbling, Hanspeter Pfister
Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy. Ternary weight…
Sparse Inter-Layer Dependencies of Transformer FFN Neurons
Johannes Knittel, Hanspeter Pfister
Feedforward network (FFN) blocks account for a large fraction of the parameters and computation in Transformer architectures, yet their internal structure remains difficult to inte…
Bias at the End of the Score
Salma Abdel Magid, Grace Guo, Esin Tureci +4
Reward models (RMs) are inherently non-neutral value functions designed and trained to encode specific objectives, such as human preferences or text-image alignment. RMs have becom…