349 citations · 1.3k across the 36 of their papers we have counts for
39 papers
Scaling Up Video Summarization Pretraining with Large Language Models
Dawit Mureja Argaw, Seunghyun Yoon, Fabian Caba Heilbron +5
Long-form video content constitutes a significant portion of internet traffic, making automated video summarization an essential research problem. However, existing video summariza…
Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation
Linzi Xing, Quan Tran, Fabian Caba +5
Video topic segmentation unveils the coarse-grained semantic structure underlying videos and is essential for other video understanding tasks. Given the recent surge in multi-modal…
DualVector: Unsupervised Vector Font Synthesis with Dual-Part Representation
Ying-Tian Liu, Zhifei Zhang, Yuan-Chen Guo +3
Automatic generation of fonts can be an important aid to typeface design. Many current approaches regard glyphs as pixelated images, which present artifacts when scaling and inevit…
Improving Diffusion Models for Scene Text Editing with Dual Encoders
Jiabao Ji, Guanhua Zhang, Zhaowen Wang +4
Scene text editing is a challenging task that involves modifying or inserting specified texts in an image while maintaining its natural and realistic appearance. Most previous appr…
Align and Attend: Multimodal Summarization with Dual Contrastive Losses
Bo He, Jun Wang, Jielin Qiu +3
The goal of multimodal summarization is to extract the most important information from different modalities to form output summaries. Unlike the unimodal summarization, the multimo…
LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos
Jielin Qiu, Franck Dernoncourt, Trung Bui +3
Livestream videos have become a significant part of online learning, where design, digital marketing, creative painting, and other skills are taught by experienced experts in the s…
Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment
Jielin Qiu, Jiacheng Zhu, Mengdi Xu +6
Multimedia summarization with multimodal output (MSMO) is a recently explored application in language grounding. It plays an essential role in real-world applications, i.e., automa…
Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition
Xudong Xie, Ling Fu, Zhifei Zhang +2
Artistic text recognition is an extremely challenging task with a wide range of applications. However, current scene text recognition methods mainly focus on irregular text while h…
MHMS: Multimodal Hierarchical Multimedia Summarization
Jielin Qiu, Jiacheng Zhu, Mengdi Xu +6
Multimedia summarization with multimodal output can play an essential role in real-world applications, i.e., automatically generating cover images and titles for news articles or p…
Font Completion and Manipulation by Cycling Between Multi-Modality Representations
Ye Yuan, Wuyang Chen, Zhaowen Wang +4
Generating font glyphs of consistent style from one or a few reference glyphs, i.e., font completion, is an important task in topographical design. As the problem is more well-defi…
Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study
Zhenyu Wu, Zhaowen Wang, Ye Yuan +3
Generative adversarial networks (GANs) nowadays are capable of producing images of incredible realism. One concern raised is whether the state-of-the-art GAN's learned distribution…
A Multi-Implicit Neural Representation for Fonts
Pradyumna Reddy, Zhifei Zhang, Matthew Fisher +3
Fonts are ubiquitous across documents and come in a variety of styles. They are either represented in a native vector format or rasterized to produce fixed resolution images. In th…
Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach
Xingqian Xu, Zhifei Zhang, Zhaowen Wang +3
Text segmentation is a prerequisite in many real-world text-related tasks, e.g., text style transfer, and scene text removal. However, facing the lack of high-quality datasets and…
Texture Hallucination for Large-Factor Painting Super-Resolution
Yulun Zhang, Zhifei Zhang, Stephen DiVerdi +3
We aim to super-resolve digital paintings, synthesizing realistic details from high-resolution reference painting materials for very large scaling factors (e.g., 8X, 16X). However,…
Large-scale Tag-based Font Retrieval with Generative Feature Learning
Tianlang Chen, Zhaowen Wang, Ning Xu +2
Font selection is one of the most important steps in a design workflow. Traditional methods rely on ordered lists which require significant domain knowledge and are often difficult…
An Internal Learning Approach to Video Inpainting
Haotian Zhang, Long Mai, Ning Xu +3
We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior'…
Controllable Artistic Text Style Transfer via Shape-Matching GAN
Shuai Yang, Zhangyang Wang, Zhaowen Wang +3
Artistic text style transfer is the task of migrating the style from a source image to the target text to create artistic typography. Recent style transfer methods have considered…
Multimodal Style Transfer via Graph Cuts
Yulun Zhang, Chen Fang, Yilin Wang +4
An assumption widely used in recent neural style transfer methods is that image styles can be described by global statics of deep features like Gram or covariance matrices. Alterna…
Dance Dance Generation: Motion Transfer for Internet Videos
Yipin Zhou, Zhaowen Wang, Chen Fang +2
This work presents computational methods for transferring body movements from one person to another with videos collected in the wild. Specifically, we train a personalized model o…
Image Super-Resolution by Neural Texture Transfer
Zhifei Zhang, Zhaowen Wang, Zhe Lin +1
Due to the significant information loss in low-resolution (LR) images, it has become extremely challenging to further advance the state-of-the-art of single image super-resolution…
Visual Font Pairing
Shuhui Jiang, Zhaowen Wang, Aaron Hertzmann +2
This paper introduces the problem of automatic font pairing. Font pairing is an important design task that is difficult for novices. Given a font selection for one part of a docume…
Learning to Sketch with Deep Q Networks and Demonstrated Strokes
Tao Zhou, Chen Fang, Zhaowen Wang +5
Doodling is a useful and common intelligent skill that people can learn and master. In this work, we propose a two-stage learning framework to teach a machine to doodle in a simula…
Flow-Grounded Spatial-Temporal Video Prediction from Still Images
Yijun Li, Chen Fang, Jimei Yang +3
Existing video prediction methods mainly rely on observing multiple historical frames or focus on predicting the next one-frame. In this work, we study the problem of generating co…
Wide Activation for Efficient and Accurate Image Super-Resolution
Jiahui Yu, Yuchen Fan, Jianchao Yang +4
In this report we demonstrate that with same parameters and computational budgets, models with wider features before ReLU activation have significantly better performance for singl…