21 citations · 28 across the 12 of their papers we have counts for
13 papers
Repetition Mismatch: Why Data Mixture Experiments Don't Scale and How to Fix Them
Kevin Zhou, Lisa Alazraki, Kris Cao +1
Pre-training data mixtures are commonly tuned by running small-scale experiments and extrapolating to the target training budget. When high-quality data is scarce and must be repea…
Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
William F. Shen, Xinchi Qiu, Chenxi Whitehouse +6
Recently, rubrics have been used to guide LLM judges in capturing subjective, nuanced, multi-dimensional human preferences, and have been extended from evaluation to reward signals…
Scaling Small Agents Through Strategy Auctions
Lisa Alazraki, William F. Shen, Yoram Bachrach +1
Small language models are increasingly viewed as a promising, cost-effective approach to agentic AI, with proponents claiming they are sufficiently capable for agentic workflows. H…
AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios
Lisa Alazraki, Lihu Chen, Ana Brassard +3
Large Language Models (LLMs) have achieved high accuracy on complex commonsense and mathematical problems that involve the composition of multiple reasoning steps. However, current…
Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection
Joe Stacey, Lisa Alazraki, Aran Ubhi +3
We investigate the robustness of fine-tuned Large Language Models (LLMs) for the task of Natural Language Inference (NLI), finding that the in-distribution gains from fine-tuning c…
Reverse Engineering Human Preferences with Reinforcement Learning
Lisa Alazraki, Tan Yi-Chern, Jon Ander Campos +3
The capabilities of Large Language Models (LLMs) are routinely evaluated by other LLMs trained to predict human preferences. This framework--known as LLM-as-a-judge--is highly scal…
Enhancing LLM Robustness to Perturbed Instructions: An Empirical Study
Aryan Agrawal, Lisa Alazraki, Shahin Honarvar +1
Large Language Models (LLMs) are highly vulnerable to input perturbations, as even a small prompt change may result in a substantially different output. Existing methods to enhance…
How can representation dimension dominate structurally pruned LLMs?
Mingxue Xu, Lisa Alazraki, Danilo P. Mandic
Pruning assumes a subnetwork exists in the original deep neural network, which can achieve comparative model performance with less computation than the original. However, it is unc…
No Need for Explanations: LLMs can implicitly learn from mistakes in-context
Lisa Alazraki, Maximilian Mozes, Jon Ander Campos +3
Showing incorrect answers to Large Language Models (LLMs) is a popular strategy to improve their performance in reasoning-intensive tasks. It is widely assumed that, in order to be…
Meta-Reasoning Improves Tool Use in Large Language Models
Lisa Alazraki, Marek Rei
External tools help large language models succeed at tasks where they would otherwise typically fail. In existing frameworks, choosing tools at test time relies on naive greedy dec…
A Multilingual Virtual Guide for Self-Attachment Technique
Alicia Jiayun Law, Ruoyu Hu, Lisa Alazraki +3
In this work, we propose a computational framework that leverages existing out-of-language data to create a conversational agent for the delivery of Self-Attachment Technique (SAT)…
How (not) to ensemble LVLMs for VQA
Lisa Alazraki, Lluis Castrejon, Mostafa Dehghani +3
This paper studies ensembling in the era of Large Vision-Language Models (LVLMs). Ensembling is a classical method to combine different models to get increased performance. In the…
An Empathetic AI Coach for Self-Attachment Therapy
Lisa Alazraki, Ali Ghachem, Neophytos Polydorou +2
In this work, we present a new dataset and a computational strategy for a digital coach that aims to guide users in practicing the protocols of self-attachment therapy. Our framewo…