123 citations · 362 across the 17 of their papers we have counts for
12 papers · 1 filter
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
Jialin Wu, Xia Hu, Yaqing Wang +2
Large multi-modal models (LMMs) exhibit remarkable performance across numerous tasks. However, generalist LMMs often suffer from performance degradation when tuned over a large col…
PaLI-3 Vision Language Models: Smaller, Faster, Stronger
Xi Chen, Xiao Wang, Lucas Beyer +16
This paper presents PaLI-3, a smaller, faster, and stronger vision language model (VLM) that compares favorably to similar models that are 10x larger. As part of arriving at this s…
PaLI-X: On Scaling up a Multilingual Vision and Language Model
Xi Chen, Josip Djolonga, Piotr Padlewski +40
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training t…
Multi-Modal Answer Validation for Knowledge-Based VQA
Jialin Wu, Jiasen Lu, Ashish Sabharwal +1
The problem of knowledge-based visual question answering involves answering questions that require external knowledge in addition to the content of the image. Such knowledge typica…
Visual Question Answering based on Local-Scene-Aware Referring Expression Generation
Jung-Jun Kim, Dong-Gyu Lee, Jialin Wu +2
Visual question answering requires a deep understanding of both images and natural language. However, most methods mainly focus on visual concept; such as the relationships between…
Improving VQA and its Explanations \\ by Comparing Competing Explanations
Jialin Wu, Liyan Chen, Raymond J. Mooney
Most recent state-of-the-art Visual Question Answering (VQA) systems are opaque black boxes that are only trained to fit the answer distribution given the question and visual conte…
Hidden State Guidance: Improving Image Captioning using An Image Conditioned Autoencoder
Jialin Wu, Raymond J. Mooney
Most RNN-based image captioning models receive supervision on the output words to mimic human captions. Therefore, the hidden states can only receive noisy gradient signals via lay…
Generating Question Relevant Captions to Aid Visual Question Answering
Jialin Wu, Zeyuan Hu, Raymond J. Mooney
Visual question answering (VQA) and image captioning require a shared body of general knowledge connecting language and vision. We present a novel approach to improve VQA performan…
Self-Critical Reasoning for Robust Visual Question Answering
Jialin Wu, Raymond J. Mooney
Visual Question Answering (VQA) deep-learning systems tend to capture superficial statistical correlations in the training data because of strong language priors and fail to genera…
Image Score: How to Select Useful Samples
Simiao Zuo, Jialin Wu
There has long been debates on how we could interpret neural networks and understand the decisions our models make. Specifically, why deep neural networks tend to be error-prone wh…
Dynamic Filtering with Large Sampling Field for ConvNets
Jialin Wu, Dai Li, Yu Yang +2
We propose a dynamic filtering strategy with large sampling field for ConvNets (LS-DFN), where the position-specific kernels learn from not only the identical position but also mul…
Action Recognition with Joint Attention on Multi-Level Deep Features
Jialin Wu, Gu Wang, Wukui Yang +1
We propose a novel deep supervised neural network for the task of action recognition in videos, which implicitly takes advantage of visual tracking and shares the robustness of bot…