5 papers · 1 filter
ZEMA: confidence-calibrated signed descent with only one state
Ali Dadsetan, Domenic Rosati, Frank Rudzicz
Recent work characterizes the Adam optimizer as sign descent modulated by gradient uncertainty: the update is the sign of the batch gradient, scaled by how strongly the batch suppo…
Multi-Token Prediction and Alternative Tokenization Improve Learning Latent Multi-Hop Graph Search at Scale
Arvid Frydenlund, FuTe Wong, Katherine Lambert +1
Given an unseen graph, a start node, , and an end node, , can a language model generate the shortest path from to ? Prior work has found that models fail to learn this…
Removing Information Content Does Not Certify Tamper Resistance in Open-Weight Models
Domenic Rosati, Alessa Carbo, Ali Dadsetan +5
Does removing harmful information make open-weight models resistant to fine-tuning attacks? We show that mutual information at release alone cannot universally certify slow recover…
Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety
Domenic Rosati, Ali Dadsetan, Hong Huang +5
A short fine-tuning run can undo the safety guards of an open-weight model---retraining a refusal-trained assistant to aid weapons development or produce hate speech. Preventing su…
Evaluating Defences against Unsafe Feedback in RLHF
Domenic Rosati, Giles Edkins, Harsh Raj +5
While there has been progress towards aligning Large Language Models (LLMs) with human values and ensuring safe behaviour at inference time, safety guards can easily be removed whe…