24 citations · 121 across the 50 of their papers we have counts for
19 papers · 1 filter
Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions
Shulin Cao, Jiajie Zhang, Jiaxin Shi +5
Large language models (LLMs) are capable of answering knowledge-intensive complex questions with chain-of-thought (CoT) reasoning. However, they tend to generate factually incorrec…
When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks
Hao Peng, Xiaozhi Wang, Jianhui Chen +8
In-context learning (ICL) has become the default method for using large language models (LLMs), making the exploration of its limitations and understanding the underlying causes cr…
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
Xiaozhi Wang, Hao Peng, Yong Guan +9
Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-eve…
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
Shangqing Tu, Yuliang Sun, Yushi Bai +3
To mitigate the potential misuse of large language models (LLMs), recent research has developed watermarking algorithms, which restrict the generation process to leave an invisible…
Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach
Zheyuan Zhang, Jifan Yu, Juanzi Li +1
Large Language Models (LLMs) have not only exhibited exceptional performance across various tasks, but also demonstrated sparks of intelligence. Recent studies have focused on asse…
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment
Ji Qi, Kaixuan Ji, Xiaozhi Wang +5
Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts, which has attracted growing attention to build dedicated models with human expe…
VidCoM: Fast Video Comprehension through Large Language Models with Multimodal Tools
Ji Qi, Kaixuan Ji, Jifan Yu +4
Building models that comprehends videos and responds specific user instructions is a practical and challenging topic, as it requires mastery of both vision understanding and knowle…
LittleMu: Deploying an Online Virtual Teaching Assistant via Heterogeneous Sources Integration and Chain of Teach Prompts
Shangqing Tu, Zheyuan Zhang, Jifan Yu +5
Teaching assistants have played essential roles in the long history of education. However, few MOOC platforms are providing human or virtual teaching assistants to support learning…
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Yushi Bai, Xin Lv, Jiajie Zhang +10
Although large language models (LLMs) demonstrate impressive performance for many language tasks, most of them can only handle texts a few thousand tokens long, limiting their appl…
VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering
Zijun Yao, Yuanyong Chen, Xin Lv +8
We present Visual Knowledge oriented Programming platform (VisKoP), a knowledge base question answering (KBQA) system that integrates human into the loop to edit and debug the know…
KoRC: Knowledge oriented Reading Comprehension Benchmark for Deep Text Understanding
Zijun Yao, Yantao Liu, Xin Lv +4
Deep text understanding, which requires the connections between a given document and prior knowledge beyond its text, has been highlighted by many benchmarks in recent years. Howev…
The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation
Hao Peng, Xiaozhi Wang, Feng Yao +5
Event extraction (EE) is a crucial task aiming at extracting events from texts, which includes two subtasks: event detection (ED) and event argument extraction (EAE). In this paper…
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Jifan Yu, Xiaozhi Wang, Shangqing Tu +32
The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations. Rather than merely exploring the breadth of LLM abilities, we believe meticu…
Benchmarking Foundation Models with Language-Model-as-an-Examiner
Yushi Bai, Jiahao Ying, Yixin Cao +10
Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability…
Learn to Not Link: Exploring NIL Prediction in Entity Linking
Fangwei Zhu, Jifan Yu, Hailong Jin +3
Entity linking models have achieved significant success via utilizing pretrained language models to capture semantic features. However, the NIL prediction problem, which aims to id…
Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering
Jiajie Zhang, Shulin Cao, Tingjia Zhang +5
Explainable question answering (XQA) aims to answer a given question and provide an explanation why the answer is selected. Existing XQA methods focus on reasoning on a single know…
MoocRadar: A Fine-grained and Multi-aspect Knowledge Repository for Improving Cognitive Student Modeling in MOOCs
Jifan Yu, Mengying Lu, Qingyang Zhong +9
Student modeling, the task of inferring a student's learning characteristics through their interactions with coursework, is a fundamental issue in intelligent education. Although t…
ChatLog: Carefully Evaluating the Evolution of ChatGPT Across Time
Shangqing Tu, Chunyang Li, Jifan Yu +3
ChatGPT has achieved great success and can be considered to have acquired an infrastructural status. There are abundant works for evaluating ChatGPT on benchmarks. However, existin…
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation
Ji Qi, Jifan Yu, Teng Tu +12
Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i.e., long and informative commen…