45 citations · 302 across the 119 of their papers we have counts for
30 papers · 2 filters
The ART of LLM Refinement: Ask, Refine, and Trust
Kumar Shridhar, Koustuv Sinha, Andrew Cohen +6
In recent years, Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations? A popular concept, referre…
Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis
David F. Jenny, Yann Billeter, Mrinmaya Sachan +2
The rapid advancement of Large Language Models (LLMs) has sparked intense debate regarding the prevalence of bias in these models and its mitigation. Yet, as exemplified by both re…
CausalCite: A Causal Formulation of Paper Citations
Ishan Kumar, Zhijing Jin, Ehsan Mokhtarian +4
Citation count of a paper is a commonly used proxy for evaluating the significance of a paper in the scientific community. Yet citation measures are widely criticized for failing t…
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
Yifan Hou, Jiaoda Li, Yu Fei +5
Recent work has shown that language models (LMs) have strong multi-step (i.e., procedural) reasoning capabilities. However, it is unclear whether LMs perform these tasks by cheatin…
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models
Ruida Wang, Wangchunshu Zhou, Mrinmaya Sachan
*Data Synthesis* is a promising way to train a small model with very little labeled data. One approach for data synthesis is to leverage the rich knowledge from large language mode…
A Diachronic Perspective on User Trust in AI under Uncertainty
Shehzaad Dhuliawala, Vilém Zouhar, Mennatallah El-Assady +1
In a human-AI collaboration, users build a mental model of the AI system based on its reliability and how it presents its decision, e.g. its presentation of system confidence and a…
Agents: An Open-source Framework for Autonomous Language Agents
Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li +15
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with e…
Tokenization and the Noiseless Channel
Vilém Zouhar, Clara Meister, Juan Luis Gastaldi +3
Subword tokenization is a key part of many NLP pipelines. However, little is known about why some tokenizer and hyperparameter combinations lead to better downstream model performa…
Adaptive and Personalized Exercise Generation for Online Language Learning
Peng Cui, Mrinmaya Sachan
Adaptive learning aims to provide customized educational activities (e.g., exercises) to address individual learning needs. However, manual construction and delivery of such activi…
A Formal Perspective on Byte-Pair Encoding
Vilém Zouhar, Clara Meister, Juan Luis Gastaldi +4
Byte-Pair Encoding (BPE) is a popular algorithm used for tokenizing data in NLP, despite being devised initially as a compression method. BPE appears to be a greedy algorithm at fa…
Can Large Language Models Infer Causation from Correlation?
Zhijing Jin, Jiarui Liu, Zhiheng Lyu +5
Causal inference is one of the hallmarks of human intelligence. While the field of CausalNLP has attracted much interest in the recent years, existing causal inference datasets in…
World Models for Math Story Problems
Andreas Opedal, Niklas Stoehr, Abulhair Saparov +1
Solving math story problems is a complex task for students and NLP models alike, requiring them to understand the world as described in the story and reason over it to compute an a…
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Yuxin Ren, Qipeng Guo, Zhijing Jin +4
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models. However, we raise a…
When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP
Jingwei Ni, Zhijing Jin, Qian Wang +2
Multi-task learning (MTL) aims at achieving a better model by leveraging data and knowledge from multiple tasks. However, MTL does not always work -- sometimes negative transfer oc…
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui +5
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the r…
Efficient Prompting via Dynamic In-Context Learning
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ryan Cotterell +1
The primary way of building AI applications is shifting from training specialist models to prompting generalist models. A common practice for prompting generalist models, often ref…
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma +3
Several recent papers claim human parity at sentence-level Machine Translation (MT), especially in high-resource languages. Thus, in response, the MT community has, in part, shifte…
Psychologically-Inspired Causal Prompts
Zhiheng Lyu, Zhijing Jin, Justus Mattern +3
NLP datasets are richer than just input-output pairs; rather, they carry causal relations between the input and output variables. In this work, we take sentiment classification as…
Membership Inference Attacks against Language Models via Neighbourhood Comparison
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin +3
Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not, and are widely used for assessing the…
Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective
Tianyu Liu, Afra Amini, Mrinmaya Sachan +1
Tasks that model the relation between pairs of tokens in a string are a vital part of understanding natural language. Such tasks, in general, require exhaustive pair-wise compariso…
A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis
Alessandro Stolfo, Yonatan Belinkov, Mrinmaya Sachan
Mathematical reasoning in large language models (LMs) has garnered significant attention in recent work, but there is a limited understanding of how these models process and store…
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems
Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury +4
While automatic dialogue tutors hold great potential in making education personalized and more accessible, research on such systems has been hampered by a lack of sufficiently larg…
Revisiting Automated Topic Model Evaluation with Large Language Models
Dominik Stammbach, Vilém Zouhar, Alexander Hoyle +2
Topic models are used to make sense of large text collections. However, automatically evaluating topic model output and determining the optimal number of topics both have been long…
Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good
Fernando Gonzalez, Zhijing Jin, Bernhard Schölkopf +3
With the recent advances in natural language processing (NLP), a vast number of applications have emerged across various use cases. Among the plethora of NLP applications, many aca…