9 papers
The Tilted Playing Field for Women in Science
Casandra Rusti, Hussain Hussain, Kian Ahrabian +4
Institutional prestige shapes access to resources, visibility, and collaboration opportunities in science. Yet whether prestige benefits researchers equally, and how it relates to…
Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization
Hasan Amin, Kian Ahrabian, Ming Yin +1
Modern language-model fine-tuning typically pairs each prompt with a single response, even though many prompts admit multiple valid completions. This effectively reduces a multi-mo…
Toward Better Temporal Structures for Geopolitical Events Forecasting
Kian Ahrabian, Eric Boxer, Jay Pujara
Forecasting on geopolitical temporal knowledge graphs (TKGs) through the lens of large language models (LLMs) has recently gained traction. While TKGs and their generalization, hyp…
A Systematic Analysis of Base Model Choice for Reward Modeling
Kian Ahrabian, Pegah Jandaghi, Negar Mokhberian +2
Reinforcement learning from human feedback (RLHF) and, at its core, reward modeling have become a crucial part of training powerful large language models (LLMs). One commonly overl…
A Practical Analysis of Human Alignment with *PO
Kian Ahrabian, Xihui Lin, Barun Patra +4
At the forefront of state-of-the-art human alignment methods are preference optimization methods (*PO). Prior research has often concentrated on identifying the best-performing met…
On The Adaptation of Unlimiformer for Decoder-Only Transformers
Kian Ahrabian, Alon Benhaim, Barun Patra +3
One of the prominent issues stifling the current generation of large language models is their limited context length. Recent proprietary models such as GPT-4 and Claude 2 have intr…