4 citations · 6 across the 5 of their papers we have counts for
5 papers
RVPO: Risk-Sensitive Alignment via Variance Regularization
Ivan Montero, Tomasz Jurczyk, Bhuwan Dhingra
Current critic-less RLHF methods aggregate multi-objective rewards via an arithmetic mean, leaving them vulnerable to constraint neglect: high-magnitude success in one objective ca…
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers
Michael Hassid, Hao Peng, Daniel Rotem +4
The attention mechanism is considered the backbone of the widely-used Transformer architecture. It contextualizes the input by computing input-specific attention matrices. We find…
Sentence Bottleneck Autoencoders from Transformer Language Models
Ivan Montero, Nikolaos Pappas, Noah A. Smith
Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems. This approach stands in contrast…
Pivot Through English: Reliably Answering Multilingual Questions without Document Retrieval
Ivan Montero, Shayne Longpre, Ni Lao +2
Existing methods for open-retrieval question answering in lower resource languages (LRLs) lag significantly behind English. They not only suffer from the shortcomings of non-Englis…
Plug and Play Autoencoders for Conditional Text Generation
Florian Mai, Nikolaos Pappas, Ivan Montero +2
Text autoencoders are commonly used for conditional generation tasks such as style transfer. We propose methods which are plug and play, where any pretrained autoencoder can be use…