78 citations · 463 across the 23 of their papers we have counts for
24 papers
OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systems
Kun Liu, Liqun Chen
The alignment of Multi-Agent Systems (MAS) for autonomous software engineering is constrained by evaluator epistemic uncertainty. Current paradigms, such as Reinforcement Learning…
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Qian Jiang, Changyou Chen, Han Zhao +6
Contrastive loss has been increasingly used in learning representations from multiple modalities. In the limit, the nature of the contrastive loss encourages modalities to exactly…
Multi-modal Alignment using Representation Codebook
Jiali Duan, Liqun Chen, Son Tran +4
Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusi…
Vision-Language Pre-Training with Triple Contrastive Learning
Jinyu Yang, Jiali Duan, Son Tran +6
Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attrib…
Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCE
Junya Chen, Zhe Gan, Xuan Li +10
InfoNCE-based contrastive representation learners, such as SimCLR, have been tremendously successful in recent years. However, these contrastive schemes are notoriously resource de…
Wasserstein Contrastive Representation Distillation
Liqun Chen, Dong Wang, Zhe Gan +3
The primary goal of knowledge distillation (KD) is to encapsulate the information of a model learned from a teacher network into a student network, with the latter being more compa…
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models
Dong Wang, Yuewei Yang, Chenyang Tao +5
Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of…
Improving Text Generation with Student-Forcing Optimal Transport
Guoyin Wang, Chunyuan Li, Jianqiao Li +10
Neural language models are often trained with maximum likelihood estimation (MLE), where the next word is generated conditioned on the ground-truth word tokens. During testing, how…
Contextualized Perturbation for Textual Adversarial Attack
Dianqi Li, Yizhe Zhang, Hao Peng +4
Adversarial examples expose the vulnerabilities of natural language processing (NLP) models, and can be used to evaluate and improve their robustness. Existing techniques of genera…
Weakly supervised cross-domain alignment with optimal transport
Siyang Yuan, Ke Bai, Liqun Chen +6
Cross-domain alignment between image objects and text sequences is key to many visual-language tasks, and it poses a fundamental challenge to both computer vision and natural langu…
Graph Optimal Transport for Cross-Domain Alignment
Liqun Chen, Zhe Gan, Yu Cheng +3
Cross-domain alignment between two sets of entities (e.g., objects in an image, words in a sentence) is fundamental to both computer vision and natural language processing. Existin…
Graph-Driven Generative Models for Heterogeneous Multi-Task Learning
Wenlin Wang, Hongteng Xu, Zhe Gan +6
We propose a novel graph-driven generative model, that unifies multiple heterogeneous learning tasks into the same framework. The proposed model is based on the fact that heterogen…
Improving Textual Network Learning with Variational Homophilic Embeddings
Wenlin Wang, Chenyang Tao, Zhe Gan +7
The performance of many network learning applications crucially hinges on the success of network embedding algorithms, which aim to encode rich network information into low-dimensi…
LMVP: Video Predictor with Leaked Motion Information
Dong Wang, Yitong Li, Wei Cao +3
We propose a Leaked Motion Video Predictor (LMVP) to predict future frames by capturing the spatial and temporal dependencies from given inputs. The motion is modeled by a newly pr…
Improving Textual Network Embedding with Global Attention via Optimal Transport
Liqun Chen, Guoyin Wang, Chenyang Tao +6
Constituting highly informative network embeddings is an important tool for network analysis. It encodes network topology, along with other useful side information, into low-dimens…
Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
Dinghan Shen, Asli Celikyilmaz, Yizhe Zhang +4
Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several…
Improving Sequence-to-Sequence Learning via Optimal Transport
Liqun Chen, Yizhe Zhang, Ruiyi Zhang +7
Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word…
Sequence Generation with Guider Network
Ruiyi Zhang, Changyou Chen, Zhe Gan +5
Sequence generation with reinforcement learning (RL) has received significant attention recently. However, a challenge with such methods is the sparse-reward problem in the RL trai…
Adversarial Text Generation via Feature-Mover's Distance
Liqun Chen, Shuyang Dai, Chenyang Tao +5
Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text…
A Unified Particle-Optimization Framework for Scalable Bayesian Sampling
Changyou Chen, Ruiyi Zhang, Wenlin Wang +2
There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-dat…
Adversarial Symmetric Variational Autoencoder
Yunchen Pu, Weiyao Wang, Ricardo Henao +4
A new form of variational autoencoder (VAE) is developed, in which the joint distribution of data and codes is considered in two (symmetric) forms: () from observed data fed thr…
Symmetric Variational Autoencoder and Connections to Adversarial Learning
Liqun Chen, Shuyang Dai, Yunchen Pu +3
A new form of the variational autoencoder (VAE) is proposed, based on the symmetric Kullback-Leibler divergence. It is demonstrated that learning of the resulting symmetric VAE (sV…
Triangle Generative Adversarial Networks
Zhe Gan, Liqun Chen, Weiyao Wang +5
A Triangle Generative Adversarial Network (-GAN) is developed for semi-supervised cross-domain joint distribution matching, where the training data consists of samples from each…
ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching
Chunyuan Li, Hao Liu, Changyou Chen +4
We investigate the non-identifiability issues associated with bidirectional adversarial training for joint distribution matching. Within a framework of conditional entropy, we prop…