2 citations · 2 across the 3 of their papers we have counts for
6 papers
Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs?
Hansi Zeng, Kai Hui, Honglei Zhuang +4
While metrics available during pre-training, such as perplexity, correlate well with model performance at scaling-laws studies, their predictive capacities at a fixed model size re…
A Watermark for Black-Box Language Models
Dara Bahri, John Wieting
Watermarking has recently emerged as an effective strategy for detecting the outputs of large language models (LLMs). Most existing schemes require white-box access to the model's…
Impact of Preference Noise on the Alignment Performance of Generative Language Models
Yang Gao, Dana Alon, Donald Metzler
A key requirement in developing Generative Language Models (GLMs) is to have their values aligned with human values. Preference-based alignment is a widely used paradigm for this p…
Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Data
Tim Baumgärtner, Yang Gao, Dana Alon +1
Reinforcement Learning from Human Feedback (RLHF) is a popular method for aligning Language Models (LM) with human values and preferences. RLHF requires a large number of preferenc…
PaRaDe: Passage Ranking using Demonstrations with Large Language Models
Andrew Drozdov, Honglei Zhuang, Zhuyun Dai +8
Recent studies show that large language models (LLMs) can be instructed to effectively perform zero-shot passage re-ranking, in which the results of a first stage retrieval method,…
OpenMSD: Towards Multilingual Scientific Documents Similarity Measurement
Yang Gao, Ji Ma, Ivan Korotkov +3
We develop and evaluate multilingual scientific documents similarity measurement models in this work. Such models can be used to find related works in different languages, which ca…