401 citations · 434 across the 12 of their papers we have counts for
14 papers
AI Research Preference Models
Thomas Simon Foster, Bassel Al Omari, Tingchen Fu +30
AI research agents (AIRA) can now carry machine learning experiments from proposal through implementation and evaluation. Yet progress on frontier tasks is throttled by the cost of…
AIRA_2: Overcoming Bottlenecks in AI Research Agents
Karen Hambardzumyan, Nicolas Baldwin, Edan Toledo +22
Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the ben…
What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
Alexis Audran-Reiss, Jordi Armengol-Estapé, Karen Hambardzumyan +17
AI research agents offer the promise to accelerate scientific progress by automating the design, implementation, and training of machine learning models. However, the field is stil…
AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench
Edan Toledo, Karen Hambardzumyan, Martin Josifoski +22
AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus o…
The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements
Bingchen Zhao, Despoina Magka, Minqi Jiang +20
Rapid advancements in large language models (LLMs) have the potential to assist in scientific progress. A critical capability toward this endeavor is the ability to reproduce exist…
With Great Backbones Comes Great Adversarial Transferability
Erik Arakelyan, Karen Hambardzumyan, Davit Papikyan +4
Advances in self-supervised learning (SSL) for machine vision have improved representation robustness and model performance, giving rise to pre-trained backbones like \emph{ResNet}…
Robust LLM safeguarding via refusal feature adversarial training
Lei Yu, Virginie Do, Karen Hambardzumyan +1
Large language models (LLMs) are vulnerable to adversarial attacks that can elicit harmful responses. Defending against such attacks remains challenging due to the opacity of jailb…
LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models
Igor Tufanov, Karen Hambardzumyan, Javier Ferrando +1
We present the LM Transparency Tool (LM-TT), an open-source interactive toolkit for analyzing the internal workings of Transformer-based language models. Differently from previousl…
Scaling Laws for Generative Mixed-Modal Language Models
Armen Aghajanyan, Lili Yu, Alexis Conneau +7
Generative language models define distributions over sequences of tokens that can represent essentially any combination of data modalities (e.g., any permutation of image tokens fr…
BARTSmiles: Generative Masked Language Models for Molecular Representations
Gayane Chilingaryan, Hovhannes Tamoyan, Ani Tevosyan +6
We discover a robust self-supervised strategy tailored towards molecular representations for generative masked language models through a series of tailored, in-depth ablations. Usi…
WARP: Word-level Adversarial ReProgramming
Karen Hambardzumyan, Hrant Khachatrian, Jonathan May
Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks. A common approach to transfer learning for multiple tasks that m…
Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks
Gor Arakelyan, Karen Hambardzumyan, Hrant Khachatrian
This paper describes our submission to CoNLL 2018 UD Shared Task. We have extended an LSTM-based neural network designed for sequence tagging to additionally generate character-lev…
Natural Language Inference over Interaction Space: ICLR 2018 Reproducibility Report
Martin Mirakyan, Karen Hambardzumyan, Hrant Khachatrian
We have tried to reproduce the results of the paper "Natural Language Inference over Interaction Space" submitted to ICLR 2018 conference as part of the ICLR 2018 Reproducibility C…
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
Nicolas Papernot, Fartash Faghri, Nicholas Carlini +23
CleverHans is a software library that provides standardized reference implementations of adversarial example construction techniques and adversarial training. The library may be us…