4 papers
Prompt Sensitivity in Tool-Augmented Model Comparisons
Mateusz Modrzejewski, Tomasz Radzikowski
Language models are compared on question-answering benchmarks that pose each question in one wording, and a reworded question can change which of two models scores higher. We ask w…
What Pairwise Evaluation Misses: Reward Scores across Prompts
Mateusz Modrzejewski, Tomasz Radzikowski
Reward models score responses for preference optimization, filtering and ranking, and pooling scores across prompts needs them to be comparable. Yet pairwise evaluation compares tw…
What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings
Mateusz Modrzejewski, Tomasz Radzikowski
Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validated by how well its ratings cor…
DINO-A: Adapting Self-Distillation Vision Transformers to General Audio Representation Learning
Tomasz Radzikowski, Mateusz Modrzejewski, Przemysław Rokita
We present DINO-A, an adaptation of self-distillation from vision to general audio representation learning. While DINO has become a canonical method in self-supervised vision and p…