16 citations · 40 across the 24 of their papers we have counts for
8 papers · 1 filter
StructTest: Benchmarking LLMs' Reasoning through Compositional Structured Outputs
Hailin Chen, Fangkai Jiao, Mathieu Ravaut +8
The rapid advancement of large language models (LLMs) demands robust, unbiased, and scalable evaluation methods. However, human annotations are costly to scale, model-based evaluat…
JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
Tong Niu, Shafiq Joty, Ye Liu +3
Accurate document retrieval is crucial for the success of retrieval-augmented generation (RAG) applications, including open-domain question answering and code completion. While lar…
P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains
Simeng Han, Aaron Yu, Rui Shen +13
Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales, which are not sufficie…
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
Yifei Ming, Senthil Purushwalkam, Shrey Pandit +4
Ensuring faithfulness to context in large language models (LLMs) and retrieval-augmented generation (RAG) systems is crucial for reliable deployment in real-world applications, as…
Direct Judgement Preference Optimization
Peifeng Wang, Austin Xu, Yilun Zhou +2
Auto-evaluation is crucial for assessing response quality and offering feedback for model development. Recent studies have explored training large language models (LLMs) as generat…
What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases
Anthony Meng Huat Tiong, Junqi Zhao, Boyang Li +3
Vision-language (VL) models, pretrained on colossal image-text datasets, have attained broad VL competence that is difficult to evaluate. A common belief is that a small number of…
A Comprehensive Survey of Contamination Detection Methods in Large Language Models
Mathieu Ravaut, Bosheng Ding, Fangkai Jiao +6
With the rise of Large Language Models (LLMs) in recent years, abundant new opportunities are emerging, but also new challenges, among which contamination is quickly becoming criti…
Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions
David Junhao Zhang, Dongxu Li, Hung Le +3
Most existing video diffusion models (VDMs) are limited to mere text conditions. Thereby, they are usually lacking in control over visual appearance and geometry structure of the g…