15 papers · 1 filter
If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs
Siqi Fan, Xiusheng Huang, Yiqun Yao +6
Large language models (LLMs) can carry out human-like dialogue, but unlike humans, they are stateless due to the superposition property. However, during multi-turn, multi-agent int…
Event Extraction in Large Language Model
Bobo Li, Xudong Han, Jiang Liu +11
Large language models (LLMs) and multimodal LLMs are changing event extraction (EE): prompting and generation can often produce structured outputs in zero shot or few shot settings…
The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models
Siqi Fan, Bowen Qin, Peng Han +3
Recent thinking models trained with reinforcement learning and backward-checking CoT often suffer from overthinking: they produce excessively long outputs even on simple problems,…
Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue History
Qishuai Zhong, Zongmin Li, Siqi Fan +1
Effective engagement by large language models (LLMs) requires adapting responses to users' sociodemographic characteristics, such as age, occupation, and education level. While man…
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks
Yixin Cao, Shibo Hong, Xinze Li +24
Large Language Models (LLMs) are advancing at an amazing speed and have become indispensable across academia, industry, and daily applications. To keep pace with the status quo, th…
FLM-101B: An Open LLM and How to Train It with $100K Budget
Xiang Li, Yiqun Yao, Xin Jiang +10
Large language models (LLMs) are considered important approaches towards foundational machine intelligence, achieving remarkable success in Natural Language Processing and multimod…