13 citations · 19 across the 16 of their papers we have counts for
18 papers
When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks
Chung-Hsiang Lo, Lu Li, Diji Yang +4
In the LLM era, many symbolic and structured problems are presented to models through 1D text serialization. Yet some such problems are natively two-dimensional: their relevant rel…
Grid2Matrix: Revealing Digital Agnosia in Vision-Language Models
Yunkai Zhang, Linda Li, Yingxin Cui +5
Vision-Language Models (VLMs) excel on many multimodal reasoning benchmarks, but these evaluations often do not require an exhaustive readout of the image and can therefore obscure…
Retrieval-Augmented Robots via Retrieve-Reason-Act
Izat Temiraliev, Diji Yang, Yi Zhang
To achieve general-purpose utility, we argue that robots must evolve from passive executors into active Information Retrieval users. In strictly zero-shot settings where no prior d…
Classroom Final Exam: An Instructor-Tested Reasoning Benchmark
Chongyang Gao, Diji Yang, Shuyan Zhou +4
We introduce CFE-Bench (Classroom Final Exam), a multimodal benchmark for evaluating the reasoning capabilities of large language models across more than 20 STEM domains. CFE-Bench…
Shared Nature, Unique Nurture: PRISM for Pluralistic Reasoning via In-context Structure Modeling
Guancheng Tu, Shiyang Zhang, Tianyu Zhang +2
Large Language Models (LLMs) are converging towards a singular Artificial Hivemind, where shared Nature (pre-training priors) result in a profound collapse of distributional divers…
VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
Haorui Yu, Diji Yang, Hang He +2
We introduce VULCA-Bench, a multicultural art-critique benchmark for evaluating Vision-Language Models' (VLMs) cultural understanding beyond surface-level visual perception. Existi…
Many Minds from One Model: Bayesian-Inspired Transformers for Population Diversity
Diji Yang, Yi Zhang
Despite their scale and success, modern transformers are usually trained as single-minded systems: optimization produces a deterministic set of parameters, representing a single fu…
Don't Waste It: Guiding Generative Recommenders with Structured Human Priors via Multi-Head Decoding
Yunkai Zhang, Qiang Zhang, Feng Lin +7
Optimizing recommender systems for objectives beyond accuracy, such as diversity, novelty, and personalization, is crucial for long-term user satisfaction. To this end, industrial…
Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing
Diji Yang, Linda Zeng, Jinmeng Rao +1
Retrieval Augmented Generation (RAG) has shown strong capability in enhancing language models' knowledge and reducing AI generative hallucinations, driving its widespread use. Howe…
GenIR: Generative Visual Feedback for Mental Image Retrieval
Diji Yang, Minghao Liu, Chung-Hsiang Lo +2
Vision-language models (VLMs) have shown strong performance on text-to-image retrieval benchmarks. However, bridging this success to real-world applications remains a challenge. In…
GRIT: Teaching MLLMs to Think with Images
Yue Fan, Xuehai He, Diji Yang +6
Recent studies have demonstrated the efficacy of using Reinforcement Learning (RL) in building reasoning models that articulate chains of thoughts prior to producing final answers.…
Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
Linda Zeng, Rithwik Gupta, Divij Motwani +2
Retrieval-augmented generation (RAG) has shown impressive capabilities in mitigating hallucinations in large language models (LLMs). However, LLMs struggle to maintain consistent r…
Beyond Introspection: Reinforcing Thinking via Externalist Behavioral Feedback
Diji Yang, Linda Zeng, Kezhen Chen +1
While inference-time thinking allows Large Language Models (LLMs) to address complex problems, the extended thinking process can be unreliable or inconsistent because of the model'…
Right this way: Can VLMs Guide Us to See More to Answer Questions?
Li Liu, Diji Yang, Sijia Zhong +4
In question-answering scenarios, humans can assess whether the available information is sufficient and seek additional information if necessary, rather than providing a forced answ…
Dual-Model Distillation for Efficient Action Classification with Hybrid Edge-Cloud Solution
Timothy Wei, Hsien Xin Peng, Elaine Xu +3
As Artificial Intelligence models, such as Large Video-Language models (VLMs), grow in size, their deployment in real-world applications becomes increasingly challenging due to har…
IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologues
Diji Yang, Jinmeng Rao, Kezhen Chen +4
Although the Retrieval-Augmented Generation (RAG) paradigms can use external knowledge to enhance and ground the outputs of Large Language Models (LLMs) to mitigate generative hall…
Tackling Vision Language Tasks Through Learning Inner Monologues
Diji Yang, Kezhen Chen, Jinmeng Rao +4
Visual language tasks require AI models to comprehend and reason with both visual and textual content. Driven by the power of Large Language Models (LLMs), two prominent methods ha…
CPL: Counterfactual Prompt Learning for Vision and Language Models
Xuehai He, Diji Yang, Weixi Feng +7
Prompt tuning is a new few-shot transfer learning technique that only tunes the learnable prompt for pre-trained vision and language models such as CLIP. However, existing prompt t…