60 citations · 143 across the 24 of their papers we have counts for
25 papers
MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration
Jiahui Peng, He Yao, Jingwen Li +9
Contrastive Language-Image Pre-training (CLIP) has demonstrated outstanding performance in global image understanding and zero-shot transfer through large-scale text-image alignmen…
SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation
Yujie Lu, Jingwen Li, Sibo Ju +5
Medical image segmentation is vital for clinical diagnosis and quantitative analysis, yet remains challenging due to the heterogeneity of imaging modalities and the high cost of pi…
TongSIM: A General Platform for Simulating Intelligent Machines
Zhe Sun, Kunlun Wu, Chuanjian Fu +24
As artificial intelligence (AI) rapidly advances, especially in multimodal large language models (MLLMs), research focus is shifting from single-modality text processing to the mor…
SIT: A Benchmark for Spatially Situated Social Intelligence Test
Zhe Sun, Xueyuan Yang, Yujie Lu +1
The integration of embodied agents into human environments demands embodied social intelligence: reasoning over both social norms and physical constraints. However, existing evalua…
Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environments
Zhenliang Zhang, Yuxi Wang, Hongzhao Xie +6
A key feature differentiating artificial general intelligence (AGI) from traditional AI is that AGI can perform composite tasks that require a wide range of capabilities. Although…
MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
Hui Chen, Miao Xiong, Yujie Lu +7
Recent advancements in AI agents have demonstrated their growing potential to drive and support scientific discovery. In this work, we introduce MLR-Bench, a comprehensive benchmar…
VITED: Video Temporal Evidence Distillation
Yujie Lu, Yale Song, William Wang +2
We investigate complex video question answering via chain-of-evidence reasoning -- identifying sequences of temporal spans from multiple relevant parts of the video, together with…
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
Xuehai He, Weixi Feng, Kaizhi Zheng +11
Multimodal Language Language Models (MLLMs) demonstrate the emerging abilities of "world models" -- interpreting and reasoning about complex real-world dynamics. To assess these ab…
WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences
Yujie Lu, Dongfu Jiang, Wenhu Chen +3
Recent breakthroughs in vision-language models (VLMs) emphasize the necessity of benchmarking human preferences in real-world multimodal interactions. To address this gap, we launc…
Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?
Xingyu Fu, Muyu He, Yujie Lu +2
We present a novel task and benchmark for evaluating the ability of text-to-image(T2I) generation models to produce images that align with commonsense in real life, which we call C…
From Text to Pixel: Advancing Long-Context Understanding in MLLMs
Yujie Lu, Xiujun Li, Tsu-Jui Fu +2
The rapid progress in Multimodal Large Language Models (MLLMs) has significantly advanced their ability to process and understand complex visual and textual information. However, t…
Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)
Michael Saxon, Fatima Jahara, Mahsa Khoshnoodi +3
With advances in the quality of text-to-image (T2I) models has come interest in benchmarking their prompt faithfulness -- the semantic coherence of generated images to the prompts…
GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks
Xinlu Zhang, Yujie Lu, Weizhi Wang +7
Automatically evaluating vision-language tasks is challenging, especially when it comes to reflecting human judgments due to limitations in accounting for fine-grained details. Alt…
Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels?
Xiujun Li, Yujie Lu, Zhe Gan +3
Recent multimodal large language models (MLLMs) have shown promising instruction following capabilities on vision-language tasks. In this work, we introduce VISUAL MODALITY INSTRUC…
Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting
Zhiyu Chen, Yujie Lu, William Yang Wang
Mental illness remains one of the most critical public health issues of our time, due to the severe scarcity and accessibility limit of professionals. Psychotherapy requires high-l…
ImagenHub: Standardizing the evaluation of conditional image generation models
Max Ku, Tianle Li, Kai Zhang +4
Recently, a myriad of conditional image generation and editing models have been developed to serve different downstream tasks, including text-to-image generation, text-guided image…
Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought
Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose +6
Despite exciting recent results showing vision-language systems' capacity to reason about images using natural language, their capacity for video reasoning remains under-explored.…
Breaking Out of the Ivory Tower: A Large-scale Analysis of Patent Citations to HCI Research
Hancheng Cao, Yujie Lu, Yuting Deng +2
What is the impact of human-computer interaction research on industry? While it is impossible to track all research impact pathways, the growing literature on translational researc…
Structured Knowledge Grounding for Question Answering
Yujie Lu, Siqi Ouyang, Kairui Zhou
Can language models (LM) ground question-answering (QA) tasks in the knowledge base via inherent relational reasoning ability? While previous models that use only LMs have seen som…
Few-Shot Document-Level Event Argument Extraction
Xianjun Yang, Yujie Lu, Linda Petzold
Event argument extraction (EAE) has been well studied at the sentence level but under-explored at the document level. In this paper, we study to capture event arguments that actual…
AstBERT: Enabling Language Model for Financial Code Understanding with Abstract Syntax Trees
Rong Liang, Tiehua Zhang, Yujie Lu +3
Using the pre-trained language models to understand source codes has attracted increasing attention from financial institutions owing to the great potential to uncover financial ri…
Federated Natural Language Generation for Personalized Dialogue System
Yujie Lu, Chao Huang, Huanli Zhan +1
Neural conversational models have long suffered from the problem of inconsistency and lacking coherent personality. To address the issue, persona-based models capturing individual…
Multi-trends Enhanced Dynamic Micro-video Recommendation
Yujie Lu, Yingxuan Huang, Shengyu Zhang +4
The explosively generated micro-videos on content sharing platforms call for recommender systems to permit personalized micro-video discovery with ease. Recent advances in micro-vi…
RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms
Wayne Xin Zhao, Shanlei Mu, Yupeng Hou +16
In recent years, there are a large number of recommendation algorithms proposed in the literature, from traditional collaborative filtering to deep learning algorithms. However, th…