132 citations · 354 across the 14 of their papers we have counts for
15 papers
Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts
Erik Daxberger, Floris Weers, Bowen Zhang +7
Sparse Mixture-of-Experts models (MoEs) have recently gained popularity due to their ability to decouple model size from inference efficiency by only activating a small subset of t…
MOFI: Learning Image Representations from Noisy Entity Annotated Images
Wentao Wu, Aleksei Timofeev, Chen Chen +8
We present MOFI, Manifold OF Images, a new vision foundation model designed to learn image representations from noisy entity annotated images. MOFI differs from previous work in tw…
STAIR: Learning Sparse Text and Image Representation in Grounded Tokens
Chen Chen, Bowen Zhang, Liangliang Cao +7
Image and text retrieval is one of the foundational tasks in the vision and language domain with multiple real-world applications. State-of-the-art approaches, e.g. CLIP, ALIGN, re…
Co-training Transformer with Videos and Images Improves Action Recognition
Bowen Zhang, Jiahui Yu, Christopher Fifty +4
In learning action recognition, models are typically pre-trained on object recognition with images, such as ImageNet, and later fine-tuned on target action recognition with videos.…
Visually Grounded Concept Composition
Bowen Zhang, Hexiang Hu, Linlu Qiu +2
We investigate ways to compose complex concepts in texts from primitive ones while grounding them in images. We propose Concept and Relation Graph (CRG), which builds on top of con…
Systematic Generalization on gSCAN: What is Nearly Solved and What is Next?
Linlu Qiu, Hexiang Hu, Bowen Zhang +2
We analyze the grounded SCAN (gSCAN) benchmark, which was recently proposed to study systematic generalization for grounded language understanding. First, we study which aspects of…
A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus
Bowen Zhang, Hexiang Hu, Joonseok Lee +5
Identifying a short segment in a long video that semantically matches a text query is a challenging task that has important application potentials in language-based video search, b…
Online Action Detection in Streaming Videos with Time Buffers
Bowen Zhang, Hao Chen, Meng Wang +1
We formulate the problem of online temporal action detection in live streaming videos, acknowledging one important property of live streaming videos that there is normally a broadc…
Learning to Represent Image and Text with Denotation Graph
Bowen Zhang, Hexiang Hu, Vihan Jain +2
Learning to fuse vision and language information and representing them is an important research problem with many applications. Recent progresses have leveraged the ideas of pre-tr…
Visual Storytelling via Predicting Anchor Word Embeddings in the Stories
Bowen Zhang, Hexiang Hu, Fei Sha
We propose a learning model for the task of visual storytelling. The main idea is to predict anchor word embeddings from the images and use the embeddings and the image features jo…
Topic Augmented Generator for Abstractive Summarization
Melissa Ailem, Bowen Zhang, Fei Sha
Steady progress has been made in abstractive summarization with attention-based sequence-to-sequence learning models. In this paper, we propose a new decoder where the output summa…
Cross-Modal and Hierarchical Modeling of Video and Text
Bowen Zhang, Hexiang Hu, Fei Sha
Visual data and text data are composed of information at multiple granularities. A video can describe a complex scene that is composed of multiple clips or shots, where each depict…
Weakly Supervised PatchNets: Describing and Aggregating Local Patches for Scene Recognition
Zhe Wang, Limin Wang, Yali Wang +2
Traditional feature encoding scheme (e.g., Fisher vector) with local descriptors (e.g., SIFT) and recent convolutional neural networks (CNNs) are two classes of successful methods…
CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016
Yuanjun Xiong, Limin Wang, Zhe Wang +7
This paper presents the method that underlies our submission to the untrimmed video classification task of ActivityNet Challenge 2016. We follow the basic pipeline of temporal segm…
Real-time Action Recognition with Enhanced Motion Vector CNNs
Bowen Zhang, Limin Wang, Zhe Wang +2
The deep two-stream architecture exhibited excellent performance on video based action recognition. The most computationally expensive step in this approach comes from the calculat…