11 papers
Selective Safety Steering via Value-Filtered Decoding
Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh +2
While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of work addresses this problem by…
How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation
Shai Feldman, Yaniv Romano
Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events -- e.g., jailb…
Testing For Distribution Shifts with Conditional Conformal Test Martingales
Shalev Shaer, Yarin Bar, Drew Prinster +1
We propose a sequential test for detecting arbitrary distribution shifts that allows conformal test martingales (CTMs) to work under a fixed, reference-conditional setting. Existin…
Semi-Supervised Hypothesis Testing by Betting on Predictions
Yaniv Tenzer, Elad Tolochinsky, Yaniv Romano
We introduce a testing-by-betting framework that leverages predictions on unlabeled data to enhance the power of sequential hypothesis testing. Given limited samples from the joint…
Learning a Continue-Thinking Token for Enhanced Test-Time Scaling
Liran Ringel, Elad Tolochinsky, Yaniv Romano
Test-time scaling has emerged as an effective approach for improving language model performance by utilizing additional compute at inference time. Recent studies have shown that ov…
Valid Best-Model Identification for LLM Evaluation via Low-Rank Factorization
Elad Tolochinsky, Yaniv Tenzer, Yaniv Romano
Selecting the best large language model (LLM) for a fixed benchmark is often expensive, since exhaustive evaluation requires running every model on every example. Multi-armed bandi…