24 citations · 100 across the 44 of their papers we have counts for
15 papers · 1 filter
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
Shangqing Tu, Yuliang Sun, Yushi Bai +3
To mitigate the potential misuse of large language models (LLMs), recent research has developed watermarking algorithms, which restrict the generation process to leave an invisible…
Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach
Zheyuan Zhang, Jifan Yu, Juanzi Li +1
Large Language Models (LLMs) have not only exhibited exceptional performance across various tasks, but also demonstrated sparks of intelligence. Recent studies have focused on asse…
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment
Ji Qi, Kaixuan Ji, Xiaozhi Wang +5
Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts, which has attracted growing attention to build dedicated models with human expe…
VidCoM: Fast Video Comprehension through Large Language Models with Multimodal Tools
Ji Qi, Kaixuan Ji, Jifan Yu +4
Building models that comprehends videos and responds specific user instructions is a practical and challenging topic, as it requires mastery of both vision understanding and knowle…
LittleMu: Deploying an Online Virtual Teaching Assistant via Heterogeneous Sources Integration and Chain of Teach Prompts
Shangqing Tu, Zheyuan Zhang, Jifan Yu +5
Teaching assistants have played essential roles in the long history of education. However, few MOOC platforms are providing human or virtual teaching assistants to support learning…
VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering
Zijun Yao, Yuanyong Chen, Xin Lv +8
We present Visual Knowledge oriented Programming platform (VisKoP), a knowledge base question answering (KBQA) system that integrates human into the loop to edit and debug the know…
KoRC: Knowledge oriented Reading Comprehension Benchmark for Deep Text Understanding
Zijun Yao, Yantao Liu, Xin Lv +4
Deep text understanding, which requires the connections between a given document and prior knowledge beyond its text, has been highlighted by many benchmarks in recent years. Howev…
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Jifan Yu, Xiaozhi Wang, Shangqing Tu +32
The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations. Rather than merely exploring the breadth of LLM abilities, we believe meticu…
Benchmarking Foundation Models with Language-Model-as-an-Examiner
Yushi Bai, Jiahao Ying, Yixin Cao +10
Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability…
Learn to Not Link: Exploring NIL Prediction in Entity Linking
Fangwei Zhu, Jifan Yu, Hailong Jin +3
Entity linking models have achieved significant success via utilizing pretrained language models to capture semantic features. However, the NIL prediction problem, which aims to id…
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction
Ji Qi, Chuchun Zhang, Xiaozhi Wang +8
The robustness to distribution changes ensures that NLP models can be successfully applied in the realistic world, especially for information extraction tasks. However, most prior…
MoocRadar: A Fine-grained and Multi-aspect Knowledge Repository for Improving Cognitive Student Modeling in MOOCs
Jifan Yu, Mengying Lu, Qingyang Zhong +9
Student modeling, the task of inferring a student's learning characteristics through their interactions with coursework, is a fundamental issue in intelligent education. Although t…
ChatLog: Carefully Evaluating the Evolution of ChatGPT Across Time
Shangqing Tu, Chunyang Li, Jifan Yu +3
ChatGPT has achieved great success and can be considered to have acquired an infrastructural status. There are abundant works for evaluating ChatGPT on benchmarks. However, existin…
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation
Ji Qi, Jifan Yu, Teng Tu +12
Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i.e., long and informative commen…
GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation
Jing Zhang, Xiaokang Zhang, Daniel Zhang-Li +10
We present GLM-Dialog, a large-scale language model (LLM) with 10B parameters capable of knowledge-grounded conversation in Chinese using a search engine to access the Internet kno…