37 citations · 74 across the 12 of their papers we have counts for
13 papers
Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
Weiming Zhuang, Jiabo Huang, Jingtao Li +4
Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between t…
Empirical Recipes for Efficient and Compact Vision-Language Models
Jiabo Huang, Zhizhong Li, Sina Sajadmanesh +2
Deploying vision-language models (VLMs) in resource-constrained settings demands low latency and high throughput, yet existing compact VLMs often fall short of the inference speedu…
StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold
Zhizhong Li, Sina Sajadmanesh, Jingtao Li +1
Low-rank adaptation (LoRA) has been widely adopted as a parameter-efficient technique for fine-tuning large-scale pre-trained models. However, it still lags behind full fine-tuning…
ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Model
Weitai Kang, Weiming Zhuang, Zhizhong Li +2
Fine-grained multimodal capability in Multimodal Large Language Models (MLLMs) has emerged as a critical research direction, particularly for tackling the visual grounding (VG) pro…
THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models
Prannay Kaul, Zhizhong Li, Hao Yang +4
Mitigating hallucinations in large vision-language models (LVLMs) remains an open problem. Recent benchmarks do not address hallucinations in open-ended free-form responses, which…
MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding
Zhanghui Kuang, Hongbin Sun, Zhizhong Li +10
We present MMOCR-an open-source toolbox which provides a comprehensive pipeline for text detection and recognition, as well as their downstream tasks such as named entity recogniti…
Representation Consolidation for Training Expert Students
Zhizhong Li, Avinash Ravichandran, Charless Fowlkes +3
Traditionally, distillation has been used to train a student model to emulate the input/output functionality of a teacher. A more useful goal than emulation, yet under-explored, is…
Regularizing Reasons for Outfit Evaluation with Gradient Penalty
Xingxing Zou, Zhizhong Li, Ke Bai +2
In this paper, we build an outfit evaluation system which provides feedbacks consisting of a judgment with a convincing explanation. The system is trained in a supervised manner wh…
Policy Continuation with Hindsight Inverse Dynamics
Hao Sun, Zhizhong Li, Xiaotong Liu +2
Solving goal-oriented tasks is an important but challenging problem in reinforcement learning (RL). For such tasks, the rewards are often sparse, making it difficult to learn a pol…
Biased Estimates of Advantages over Path Ensembles
Lanxin Lei, Zhizhong Li, Dahua Lin
The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family o…
Improving Confidence Estimates for Unfamiliar Examples
Zhizhong Li, Derek Hoiem
Intuitively, unfamiliarity should lead to lack of confidence. In reality, current algorithms often make highly confident yet wrong predictions when faced with relevant but unfamili…
Integrating Specialized Classifiers Based on Continuous Time Markov Chain
Zhizhong Li, Dahua Lin
Specialized classifiers, namely those dedicated to a subset of classes, are often adopted in real-world recognition systems. However, integrating such classifiers is nontrivial. Ex…
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
Xingcheng Zhang, Zhizhong Li, Chen Change Loy +1
A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition. In our study, h…