7 citations · 10 across the 9 of their papers we have counts for
10 papers
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
Shijie Zhou, Viet Dac Lai, Hao Tan +4
Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regions on the screen. Existing Multimodal…
Multimodal LLMs as Customized Reward Models for Text-to-Image Generation
Shijie Zhou, Ruiyi Zhang, Huaisheng Zhu +5
We introduce LLaVA-Reward, an efficient reward model designed to automatically evaluate text-to-image (T2I) generations across multiple perspectives, leveraging pretrained multimod…
MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models
Jian Chen, Wenye Ma, Penghang Liu +7
Multimodal Large Language Models (MLLMs) have achieved remarkable visual reasoning abilities in natural images, text-rich documents, and graphic designs. However, their ability to…
A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction Generation
Shijie Zhou, Ruiyi Zhang, Yufan Zhou +1
Large multimodal models still struggle with text-rich images because of inadequate training data. Self-Instruct provides an annotation-free way for generating instruction data, but…
Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements
Shijie Zhou, Huaisheng Zhu, Rohan Sharma +4
Diffusion models have emerged as a powerful foundation model for visual generations. With an appropriate sampling process, it can effectively serve as a generative prior for solvin…
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Jian Chen, Ruiyi Zhang, Yufan Zhou +4
Controllable layout generation refers to the process of creating a plausible visual arrangement of elements within a graphic design (e.g., document and web designs) with constraint…
STT: Soft Template Tuning for Few-Shot Adaptation
Ping Yu, Wei Wang, Chunyuan Li +3
Prompt tuning has been an extremely effective tool to adapt a pre-trained model to downstream tasks. However, standard prompt-based methods mainly consider the case of sufficient d…
SDA: Improving Text Generation with Self Data Augmentation
Ping Yu, Ruiyi Zhang, Yang Zhao +3
Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision. However, less work has been done in the context of text, pa…
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
Bang An, Jie Lyu, Zhenyi Wang +6
The neural attention mechanism plays an important role in many natural language processing applications. In particular, the use of multi-head attention extends single-head attentio…
Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions
Zhenyi Wang, Ping Yu, Yang Zhao +4
Human-motion generation is a long-standing challenging task due to the requirement of accurately modeling complex and diverse dynamic patterns. Most existing methods adopt sequence…