14 citations · 39 across the 24 of their papers we have counts for
24 papers
SynerGen: Contextualized Generative Recommender for Unified Search and Recommendation
Vianne R. Gao, Chen Xue, Marc Versage +11
The dominant retrieve-then-rank pipeline in large-scale recommender systems suffers from mis-calibration and engineering overhead due to its architectural split and differing optim…
Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval
Yuxiang Liu, Tian Wang, Gourab Kundu +6
Transformer-based models such as BERT and E5 have significantly advanced text embedding by capturing rich contextual representations. However, many complex real-world queries requi…
AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs
Nicholas E. Corrado, Julian Katz-Samuels, Adithya Devraj +6
When aligning large language models (LLMs), their performance on various tasks (such as being helpful, harmless, and honest) depends heavily on the composition of their training da…
InfoPO: On Mutual Information Maximization for Large Language Model Alignment
Teng Xiao, Zhen Ge, Sujay Sanghavi +5
We study the post-training of large language models (LLMs) with human preference data. Recently, direct preference optimization and its variants have shown considerable promise in…
M-LLM Based Video Frame Selection for Efficient Video Understanding
Kai Hu, Feng Gao, Xiaohan Nie +8
Recent advances in Multi-Modal Large Language Models (M-LLMs) show promising results in video reasoning. Popular Multi-Modal Large Language Model (M-LLM) frameworks usually apply n…
CoLLM: A Large Language Model for Composed Image Retrieval
Chuong Huynh, Jinyu Yang, Ashish Tawari +5
Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query. Typical training data consists of triplets containing a reference image,…
DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Models
Shwetha Ram, Tal Neiman, Qianli Feng +3
Given a small number of images of a subject, personalized image generation techniques can fine-tune large pre-trained text-to-image diffusion models to generate images of the subje…
Evolutionary Contrastive Distillation for Language Model Alignment
Julian Katz-Samuels, Zheng Li, Hyokun Yun +5
The ability of large language models (LLMs) to execute complex instructions is essential for their real-world applications. However, several recent studies indicate that LLMs strug…
X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs
Sirnam Swetha, Jinyu Yang, Tal Neiman +5
Recent advancements in Multimodal Large Language Models (MLLMs) have revolutionized the field of vision-language understanding by integrating visual perception capabilities into La…
Open Vocabulary Multi-Label Video Classification
Rohit Gupta, Mamshad Nayeem Rizve, Jayakrishnan Unnikrishnan +5
Pre-trained vision-language models (VLMs) have enabled significant progress in open vocabulary computer vision tasks such as image classification, object detection and image segmen…
VidLA: Video-Language Alignment at Scale
Mamshad Nayeem Rizve, Fan Fei, Jayakrishnan Unnikrishnan +5
In this paper, we propose VidLA, an approach for video-language alignment at scale. There are two major limitations of previous video-language alignment approaches. First, they do…
Robust Multi-Task Learning with Excess Risks
Yifei He, Shiji Zhou, Guojun Zhang +5
Multi-task learning (MTL) considers learning a joint model for multiple tasks by optimizing a convex combination of all task losses. To solve the optimization problem, existing met…
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Han Xie, Da Zheng, Jun Ma +9
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain. In the graph mining domain, a similar analogy can be…
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Qian Jiang, Changyou Chen, Han Zhao +6
Contrastive loss has been increasingly used in learning representations from multiple modalities. In the limit, the nature of the contrastive loss encourages modalities to exactly…
SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
Chaoyang He, Shuai Zheng, Aston Zhang +4
The mixture of Expert (MoE) parallelism is a recent advancement that scales up the model size with constant computational cost. MoE selects different sets of parameters (i.e., expe…
MICO: Selective Search with Mutual Information Co-training
Zhanyu Wang, Xiao Zhang, Hyokun Yun +2
In contrast to traditional exhaustive search, selective search first clusters documents into several groups before all the documents are searched exhaustively by a query, to limit…
Efficient and effective training of language and graph neural network models
Vassilis N. Ioannidis, Xiang Song, Da Zheng +6
Can we combine heterogenous graph structure with text to learn high-quality semantic and behavioural representations? Graph neural networks (GNN)s encode numerical node attributes…
DynaMaR: Dynamic Prompt with Mask Token Representation
Xiaodi Sun, Sunny Rajagopalan, Priyanka Nigam +4
Recent research has shown that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks. Typically wh…
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud
Zhen Zhang, Shuai Zheng, Yida Wang +5
Existing general purpose frameworks for gigantic model training, i.e., dense models with billions of parameters, cannot scale efficiently on cloud environment with various networki…
Multi-modal Alignment using Representation Codebook
Jiali Duan, Liqun Chen, Son Tran +4
Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusi…
Vision-Language Pre-Training with Triple Contrastive Learning
Jinyu Yang, Jiali Duan, Son Tran +6
Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attrib…
Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
Xuanli He, Iman Keivanloo, Yi Xu +4
Pre-training and then fine-tuning large language models is commonly used to achieve state-of-the-art performance in natural language processing (NLP) tasks. However, most pre-train…
MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling
Tarik Arici, Mehmet Saygin Seyfioglu, Tal Neiman +5
Vision-and-Language Pre-training (VLP) improves model performance for downstream tasks that require image and text inputs. Current VLP approaches differ on (i) model architecture (…
Tiering as a Stochastic Submodular Optimization Problem
Hyokun Yun, Michael Froh, Roshan Makhijani +3
Tiering is an essential technique for building large-scale information retrieval systems. While the selection of documents for high priority tiers critically impacts the efficiency…