6 papers
CodeSimpleQA: Scaling Factuality in Code Large Language Models
Jian Yang, Wei Zhang, Yizhi Li +8
Large language models (LLMs) have made significant strides in code generation, achieving impressive capabilities in synthesizing code snippets from natural language instructions. H…
Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models
Meng Cao, Pengfei Hu, Yingyao Wang +12
Recent advancements in Large Video Language Models (LVLMs) have highlighted their potential for multi-modal understanding, yet evaluating their factual grounding in videos remains…
IFEvalCode: Controlled Code Generation
Jian Yang, Wei Zhang, Shukai Liu +9
Code large language models (Code LLMs) have made significant progress in code generation by translating natural language descriptions into functional code; however, real-world appl…
Reinforcement learning fine-tuning of language model for instruction following and math reasoning
Yifu Han, Geo Zhang
This study investigates the effectiveness of reinforcement learning (RL) fine-tuning techniques on a compact language model (Qwen2.5-0.5B Base) for two challenging tasks: instructi…
Clarifying Before Reasoning: A Coq Prover with Structural Context
Yanzhen Lu, Hanbin Yang, Xiaodie Wang +6
In this work, we investigate whether improving task clarity can enhance reasoning ability of large language models, focusing on theorem proving in Coq. We introduce a concept-level…
LIFEBench: Evaluating Length Instruction Following in Large Language Models
Wei Zhang, Zhenhong Zhou, Kun Wang +9
While large language models (LLMs) can solve PhD-level reasoning problems over long context inputs, they still struggle with a seemingly simpler task: following explicit length ins…