151 citations · 348 across the 25 of their papers we have counts for
26 papers
Modeling Distinct Human Interaction in Web Agents
Faria Huq, Zora Zhiruo Wang, Zhanqiu Guo +6
Despite rapid progress in autonomous web agents, human involvement remains essential for shaping preferences and correcting agent behavior as tasks unfold. However, current agentic…
OpenHands: An Open Platform for AI Software Developers as Generalist Agents
Xingyao Wang, Boxuan Li, Yufan Song +21
Software is one of the most powerful tools that we humans have at our disposal; it allows a skilled programmer to interact with the world in complex and profound ways. At the same…
CodeRAG-Bench: Can Retrieval Augment Code Generation?
Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu +4
While language models (LMs) have proven remarkably adept at generating code, many programs are challenging for LMs to generate using their parametric knowledge alone. Providing ext…
WebArena: A Realistic Web Environment for Building Autonomous Agents
Shuyan Zhou, Frank F. Xu, Hao Zhu +9
With advances in generative AI, there is now potential for autonomous agents to manage daily tasks via natural language commands. However, current agents are primarily created and…
Hierarchical Prompting Assists Large Language Model on Web Navigation
Abishek Sridhar, Robert Lo, Frank F. Xu +2
Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks. To alleviate this issue, we propose a simple hierarchical prompti…
Active Retrieval Augmented Generation
Zhengbao Jiang, Frank F. Xu, Luyu Gao +6
Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augme…
Why do Nearest Neighbor Language Models Work?
Frank F. Xu, Uri Alon, Graham Neubig
Language models (LMs) compute the probability of a text by sequentially computing a representation of an already-seen context and using this representation to predict the next word…
DocPrompting: Generating Code by Retrieving the Docs
Shuyan Zhou, Uri Alon, Frank F. Xu +3
Publicly available source-code libraries are continuously growing and changing. This makes it impossible for models of code to keep current with all available APIs by simply traini…
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages
Zhiruo Wang, Grace Cuenca, Shuyan Zhou +2
While there has been a recent burgeoning of applications at the intersection of natural and programming languages, such as code generation and code summarization, these application…
A Systematic Evaluation of Large Language Models of Code
Frank F. Xu, Uri Alon, Graham Neubig +1
Large language models (LMs) of code have recently shown tremendous promise in completing code and synthesizing code from natural language descriptions. However, the current state-o…
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval
Uri Alon, Frank F. Xu, Junxian He +3
Retrieval-based language models (R-LM) model the probability of natural language text by combining a standard language model (LM) with examples retrieved from an external datastore…
Capturing Structural Locality in Non-parametric Language Models
Frank F. Xu, Junxian He, Graham Neubig +1
Structural locality is a ubiquitous feature of real-world datasets, wherein data points are organized into local hierarchies. Some examples include topical clusters in text or proj…
Learning Structural Edits via Incremental Tree Transformations
Ziyu Yao, Frank F. Xu, Pengcheng Yin +2
While most neural generative models generate outputs in a single pass, the human creative process is usually one of iterative building and refinement. Recent work has proposed mode…
In-IDE Code Generation from Natural Language: Promise and Challenges
Frank F. Xu, Bogdan Vasilescu, Graham Neubig
A great part of software development involves conceptualizing or communicating the underlying procedures and logic that needs to be expressed in programs. One major difficulty of p…
A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos
Frank F. Xu, Lei Ji, Botian Shi +4
Watching instructional videos are often used to learn about procedures. Video captioning is one way of automatically collecting such knowledge. However, it provides only an indirec…
Minimally Supervised Categorization of Text with Metadata
Yu Zhang, Yu Meng, Jiaxin Huang +3
Document categorization, which aims to assign a topic label to each document, plays a fundamental role in a wide variety of applications. Despite the success of existing studies in…
Incorporating External Knowledge through Pre-training for Natural Language to Code Generation
Frank F. Xu, Zhengbao Jiang, Pengcheng Yin +2
Open-domain code generation aims to generate code in a general-purpose programming language (such as Python) from natural language (NL) intents. Motivated by the intuition that dev…
How Can We Know What Language Models Know?
Zhengbao Jiang, Frank F. Xu, Jun Araki +1
Recent work has presented intriguing results examining the knowledge contained in language models (LM) by having the LM fill in the blanks of prompts such as "Obama is a _ by profe…
HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories
Yu Zhang, Frank F. Xu, Sha Li +4
GitHub has become an important platform for code sharing and scientific exchange. With the massive number of repositories available, there is a pressing need for topic-based search…
StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible
Anhong Guo, Junhan Kong, Michael Rivera +2
Blind people frequently encounter inaccessible dynamic touchscreens in their everyday lives that are difficult, frustrating, and often impossible to use independently. Touchscreens…
Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings
Ahmed El-Kishky, Frank Xu, Aston Zhang +1
Traditionally, many text-mining tasks treat individual word-tokens as the finest meaningful semantic granularity. However, in many languages and specialized corpora, words are comp…
Data-to-Text Generation with Style Imitation
Shuai Lin, Wentao Wang, Zichao Yang +4
Recent neural approaches to data-to-text generation have mostly focused on improving content fidelity while lacking explicit control over writing styles (e.g., word choices, senten…
Integrating Local Context and Global Cohesiveness for Open Information Extraction
Qi Zhu, Xiang Ren, Jingbo Shang +3
Extracting entities and their relations from text is an important task for understanding massive text corpora. Open information extraction (IE) systems mine relation tuples (i.e.,…
Indirect Supervision for Relation Extraction using Question-Answer Pairs
Zeqiu Wu, Xiang Ren, Frank F. Xu +2
Automatic relation extraction (RE) for types of interest is of great importance for interpreting massive text corpora in an efficient manner. Traditional RE models have heavily rel…