112 citations · 533 across the 82 of their papers we have counts for
13 papers · 2 filters
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Xiang Yue, Yuansheng Ni, Kai Zhang +19
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU…
Multimodal Question Answering for Unified Information Extraction
Yuxuan Sun, Kai Zhang, Yu Su
Multimodal information extraction (MIE) aims to extract structured information from unstructured multimedia content. Due to the diversity of tasks and settings, most current MIE mo…
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
Xiang Yue, Xingwei Qu, Ge Zhang +5
We introduce MAmmoTH, a series of open-source large language models (LLMs) specifically tailored for general math problem-solving. The MAmmoTH models are trained on MathInstruct, o…
Roll Up Your Sleeves: Working with a Collaborative and Engaging Task-Oriented Dialogue System
Lingbo Mo, Shijie Chen, Ziru Chen +10
We introduce TacoBot, a user-centered task-oriented digital assistant designed to guide users through complex real-world tasks with multiple steps. Covering a wide range of cooking…
Biomedical Language Models are Robust to Sub-optimal Tokenization
Bernal Jiménez Gutiérrez, Huan Sun, Yu Su
As opposed to general English, many concepts in biomedical terminology have been designed in recent history by biomedical professionals with the goal of being precise and concise.…
Mind2Web: Towards a Generalist Agent for the Web
Xiang Deng, Yu Gu, Boyuan Zheng +5
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website.…
Text-to-SQL Error Correction with Language Models of Code
Ziru Chen, Shijie Chen, Michael White +5
Despite recent progress in text-to-SQL parsing, current semantic parsers are still not accurate enough for practical use. In this paper, we investigate how to build automatic text-…
Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms
Tianshu Zhang, Changchang Liu, Wei-Han Lee +2
This paper studies a new task of federated learning (FL) for semantic parsing, where multiple clients collaboratively train one global model without sharing their semantic parsing…
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors
Kai Zhang, Bernal Jiménez Gutiérrez, Yu Su
Recent work has shown that fine-tuning large language models (LLMs) on large-scale instruction-following datasets substantially improves their performance on a wide range of NLP ta…
Few-shot In-context Learning for Knowledge Base Question Answering
Tianle Li, Xueguang Ma, Alex Zhuang +3
Question answering over knowledge bases is considered a difficult problem due to the challenge of generalizing to a wide variety of possible natural language questions. Additionall…
Error Detection for Text-to-SQL Semantic Parsing
Shijie Chen, Ziru Chen, Huan Sun +1
Despite remarkable progress in text-to-SQL semantic parsing in recent years, the performance of existing parsers is still far from perfect. Specifically, modern text-to-SQL parsers…
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
Jian Xie, Kai Zhang, Jiangjie Chen +2
By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limit…
Automatic Evaluation of Attribution by Large Language Models
Xiang Yue, Boshi Wang, Ziru Chen +3
A recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external references to generate and support its claims. How…