244 citations · 966 across the 45 of their papers we have counts for
51 papers
Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers
Chongjian Ge, Hanwen Jiang, Tianyu Wang +9
Visual generation increasingly requires high-resolution images, long videos, and multimodal context, making the quadratic cost of full attention prohibitive. We introduce Chimera,…
SimpleMatch: A Simple and Strong Baseline for Semantic Correspondence
Hailing Jin, Huiying Li
Recent advances in semantic correspondence have been largely driven by the use of pre-trained large-scale models. However, a limitation of these approaches is their dependence on h…
Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training
Dezhao Luo, Jiabo Huang, Shaogang Gong +2
The correlation between the vision and text is essential for video moment retrieval (VMR), however, existing methods heavily rely on separate pre-training feature extractors for vi…
LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos
Jielin Qiu, Franck Dernoncourt, Trung Bui +3
Livestream videos have become a significant part of online learning, where design, digital marketing, creative painting, and other skills are taught by experienced experts in the s…
Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment
Jielin Qiu, Jiacheng Zhu, Mengdi Xu +6
Multimedia summarization with multimodal output (MSMO) is a recently explored application in language grounding. It plays an essential role in real-world applications, i.e., automa…
Video Activity Localisation with Uncertainties in Temporal Boundary
Jiabo Huang, Hailin Jin, Shaogang Gong +1
Current methods for video activity localisation over time assume implicitly that activity temporal boundaries labelled for model training are determined and precise. However, in un…
MHMS: Multimodal Hierarchical Multimedia Summarization
Jielin Qiu, Jiacheng Zhu, Mengdi Xu +6
Multimedia summarization with multimodal output can play an essential role in real-world applications, i.e., automatically generating cover images and titles for news articles or p…
StyleBabel: Artistic Style Tagging and Captioning
Dan Ruta, Andrew Gilbert, Pranav Aggarwal +9
We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a nov…
Time-Equivariant Contrastive Video Representation Learning
Simon Jenni, Hailin Jin
We introduce a novel self-supervised contrastive learning method to learn representations from unlabelled videos. Existing approaches ignore the specifics of input distortions, e.g…
Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos
Reuben Tan, Bryan A. Plummer, Kate Saenko +2
We introduce the task of spatially localizing narrated interactions in videos. Key to our approach is the ability to learn to spatially localize interactions with self-supervision…
Cross Modal Retrieval with Querybank Normalisation
Simion-Vlad Bogolin, Ioana Croitoru, Hailin Jin +2
Profiting from large-scale training datasets, advances in neural architecture design and efficient inference, joint embeddings have become the dominant approach for tackling cross-…
StreamHover: Livestream Transcript Summarization and Annotation
Sangwoo Cho, Franck Dernoncourt, Tim Ganter +7
With the explosive growth of livestream broadcasting, there is an urgent need for new summarization technology that enables us to create a preview of streamed content and tap into…
Font Completion and Manipulation by Cycling Between Multi-Modality Representations
Ye Yuan, Wuyang Chen, Zhaowen Wang +4
Generating font glyphs of consistent style from one or a few reference glyphs, i.e., font completion, is an important task in topographical design. As the problem is more well-defi…
Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation
Jiabo Huang, Yang Liu, Shaogang Gong +1
Video activity localisation has recently attained increasing attention due to its practical values in automatically localising the most salient visual segments corresponding to the…
Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study
Zhenyu Wu, Zhaowen Wang, Ye Yuan +3
Generative adversarial networks (GANs) nowadays are capable of producing images of incredible realism. One concern raised is whether the state-of-the-art GAN's learned distribution…
Compositional Sketch Search
Alexander Black, Tu Bui, Long Mai +2
We present an algorithm for searching image collections using free-hand sketches that describe the appearance and relative positions of multiple objects. Sketch based image retriev…
Magic Layouts: Structural Prior for Component Detection in User Interface Designs
Dipu Manandhar, Hailin Jin, John Collomosse
We present Magic Layouts; a method for parsing screenshots or hand-drawn sketches of user interface (UI) layouts. Our core contribution is to extend existing detectors to exploit a…
A Multi-Implicit Neural Representation for Fonts
Pradyumna Reddy, Zhifei Zhang, Matthew Fisher +3
Fonts are ubiquitous across documents and come in a variety of styles. They are either represented in a native vector format or rasterized to produce fixed resolution images. In th…
TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrieval
Ioana Croitoru, Simion-Vlad Bogolin, Marius Leordeanu +4
In recent years, considerable progress on the task of text-video retrieval has been achieved by leveraging large-scale pretraining on visual and audio datasets to construct powerfu…
ALADIN: All Layer Adaptive Instance Normalization for Fine-grained Style Similarity
Dan Ruta, Saeid Motiian, Baldo Faieta +5
We present ALADIN (All Layer AdaIN); a novel architecture for searching images based on the similarity of their artistic style. Representation learning is critical to visual search…
Video Question Answering on Screencast Tutorials
Wentian Zhao, Seokhwan Kim, Ning Xu +1
This paper presents a new video question answering task on screencast tutorials. We introduce a dataset including question, answer and context triples from the tutorial videos for…
Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction
Tong He, John Collomosse, Hailin Jin +1
We propose Geo-PIFu, a method to recover a 3D mesh from a monocular color image of a clothed person. Our method is based on a deep implicit function-based representation to learn l…
Steering Self-Supervised Feature Learning Beyond Local Pixel Statistics
Simon Jenni, Hailin Jin, Paolo Favaro
We introduce a novel principle for self-supervised feature learning based on the discrimination of specific transformations of an image. We argue that the generalization capability…
Superpixel Segmentation with Fully Convolutional Networks
Fengting Yang, Qian Sun, Hailin Jin +1
In computer vision, superpixels have been widely used as an effective way to reduce the number of image primitives for subsequent processing. But only a few attempts have been made…