14 citations · 42 across the 18 of their papers we have counts for
20 papers
Native Action-Prior Learning from Videos for World Action Models
Zhaochong An, Fei Zhang, Menglin Jia +10
World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action-annotated robot trajectories. Observ…
Unifying Video Tasks via Spatiotemporal Analogy
Chia-Hsiang Kao, Belinda Zeng, Bharath Hariharan +1
Adapting video models to new tasks typically requires dedicated data curation and fine-tuning. While visual analogy provides a training-free alternative by specifying tasks in-cont…
TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents
Shoufa Chen, Luyuan Wang, Xuan Yang +7
As large language models and harness frameworks continue to advance, agents operating in terminals are increasingly capable of performing a broader range of general computer-use ta…
WavFlow: Audio Generation in Waveform Space
Feiyan Zhou, Luyuan Wang, Shoufa Chen +6
Modern audio generation predominantly relies on latent-space compression, introducing additional complexity and potential information loss. In this work, we challenge this paradigm…
Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation
Zhiheng Liu, Weiming Ren, Xiaoke Huang +12
Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the t…
Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings
Xuanqing Liu, Luyang Kong, Wei Niu +6
Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. However, analyzing live d…
Diffusion Models For Multi-Modal Generative Modeling
Changyou Chen, Han Ding, Bunyamin Sisman +5
Diffusion-based generative modeling has been achieving state-of-the-art results on various generation tasks. Most diffusion models, however, are limited to a single-generation mode…
VidLA: Video-Language Alignment at Scale
Mamshad Nayeem Rizve, Fan Fei, Jayakrishnan Unnikrishnan +5
In this paper, we propose VidLA, an approach for video-language alignment at scale. There are two major limitations of previous video-language alignment approaches. First, they do…
Robust Multi-Task Learning with Excess Risks
Yifei He, Shiji Zhou, Guojun Zhang +5
Multi-task learning (MTL) considers learning a joint model for multiple tasks by optimizing a convex combination of all task losses. To solve the optimization problem, existing met…
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective
Yue Xing, Xiaofeng Lin, Qifan Song +3
Pre-training is known to generate universal representations for downstream tasks in large-scale deep learning such as large language models. Existing literature, e.g., \cite{kim202…
ForeSeer: Product Aspect Forecasting Using Temporal Graph Embedding
Zixuan Liu, Gaurush Hiranandani, Kun Qian +5
Developing text mining approaches to mine aspects from customer reviews has been well-studied due to its importance in understanding customer needs and product attributes. In contr…
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Han Xie, Da Zheng, Jun Ma +9
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain. In the graph mining domain, a similar analogy can be…
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Qian Jiang, Changyou Chen, Han Zhao +6
Contrastive loss has been increasingly used in learning representations from multiple modalities. In the limit, the nature of the contrastive loss encourages modalities to exactly…
Efficient and effective training of language and graph neural network models
Vassilis N. Ioannidis, Xiang Song, Da Zheng +6
Can we combine heterogenous graph structure with text to learn high-quality semantic and behavioural representations? Graph neural networks (GNN)s encode numerical node attributes…
DynaMaR: Dynamic Prompt with Mask Token Representation
Xiaodi Sun, Sunny Rajagopalan, Priyanka Nigam +4
Recent research has shown that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks. Typically wh…
Multi-modal Alignment using Representation Codebook
Jiali Duan, Liqun Chen, Son Tran +4
Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusi…
Vision-Language Pre-Training with Triple Contrastive Learning
Jinyu Yang, Jiali Duan, Son Tran +6
Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attrib…
Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
Xuanli He, Iman Keivanloo, Yi Xu +4
Pre-training and then fine-tuning large language models is commonly used to achieve state-of-the-art performance in natural language processing (NLP) tasks. However, most pre-train…
MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling
Tarik Arici, Mehmet Saygin Seyfioglu, Tal Neiman +5
Vision-and-Language Pre-training (VLP) improves model performance for downstream tasks that require image and text inputs. Current VLP approaches differ on (i) model architecture (…
Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCE
Junya Chen, Zhe Gan, Xuan Li +10
InfoNCE-based contrastive representation learners, such as SimCLR, have been tremendously successful in recent years. However, these contrastive schemes are notoriously resource de…