5 citations · 15 across the 32 of their papers we have counts for
17 papers · 1 filter
Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant"
Yudong Zhang, Ruobing Xie, Jiansheng Chen +3
Contrastive learning is a prevalent technique in self-supervised vision representation learning, typically generating positive pairs by applying two data augmentations to the same…
Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
Yuhan Fu, Ruobing Xie, Xingwu Sun +2
Multimodal Large Language Models (MLLMs) are known to hallucinate, which limits their practical applications. Recent works have attempted to apply Direct Preference Optimization (D…
DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models
Yudong Zhang, Ruobing Xie, Xingwu Sun +5
Large vision-language models (LVLMs) have demonstrated exceptional performance on complex multimodal tasks. However, they continue to suffer from significant hallucination issues,…
More Expressive Attention with Negative Weights
Ang Lv, Ruobing Xie, Shuaipeng Li +5
We propose a novel attention mechanism, named Cog Attention, that enables attention weights to be negative for enhanced expressiveness, which stems from two key factors: (1) Cog At…
Exploring Forgetting in Large Language Model Pre-Training
Chonghua Liao, Ruobing Xie, Xingwu Sun +2
Catastrophic forgetting remains a formidable obstacle to building an omniscient model in large language models (LLMs). Despite the pioneering research on task-level forgetting in L…
Lossless KV Cache Compression to 2%
Zhen Yang, J. N. Han, Kan Wu +4
Large language models have revolutionized data processing in numerous domains, with their ability to handle extended context reasoning receiving notable recognition. To speed up in…
RosePO: Aligning LLM-based Recommenders with Human Values
Jiayi Liao, Xiangnan He, Ruobing Xie +5
Recently, there has been a growing interest in leveraging Large Language Models (LLMs) for recommendation systems, which usually adapt a pre-trained LLM to the recommendation scena…
Continuous Speech Tokenizer in Text To Speech
Yixing Li, Ruobing Xie, Xingwu Sun +2
The fusion of speech and language in the era of large language models has garnered significant attention. Discrete speech token is often utilized in text-to-speech tasks for speech…
Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions
Yuhan Fu, Ruobing Xie, Jiazhen Liu +4
Hallucinations in multimodal large language models (MLLMs) hinder their practical applications. To address this, we propose a Magnifier Prompt (MagPrompt), a simple yet effective m…
PIP: Detecting Adversarial Examples in Large Vision-Language Models via Attention Patterns of Irrelevant Probe Questions
Yudong Zhang, Ruobing Xie, Jiansheng Chen +2
Large Vision-Language Models (LVLMs) have demonstrated their powerful multimodal capabilities. However, they also face serious safety problems, as adversaries can induce robustness…
Language Models "Grok" to Copy
Ang Lv, Ruobing Xie, Xingwu Sun +2
We examine the pre-training dynamics of language models, focusing on their ability to copy text from preceding context--a fundamental skill for various LLM applications, including…
Negative Sampling in Recommendation: A Survey and Future Directions
Haokai Ma, Ruobing Xie, Lei Meng +5
Recommender system (RS) aims to capture personalized preferences from massive user behaviors, making them pivotal in the era of information explosion. However, the presence of ``in…
HMoE: Heterogeneous Mixture of Experts for Language Modeling
An Wang, Xingwu Sun, Ruobing Xie +9
Mixture of Experts (MoE) offers remarkable performance and computational efficiency by selectively activating subsets of model parameters. Traditionally, MoE models use homogeneous…
Diverse and Fine-Grained Instruction-Following Ability Exploration with Synthetic Data
Zihui Gu, Xingwu Sun, Fengzong Lian +3
Instruction-following is particularly crucial for large language models (LLMs) to support diverse user requests. While existing work has made progress in aligning LLMs with human p…
Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling
Shuaipeng Li, Penghao Zhao, Hailin Zhang +10
In current deep learning tasks, Adam style optimizers such as Adam, Adagrad, RMSProp, Adafactor, and Lion have been widely used as alternatives to SGD style optimizers. These optim…
PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
Jiazhen Liu, Yuhan Fu, Ruobing Xie +5
Multimodal Large Language Models (MLLMs) hallucinate, resulting in an emerging topic of visual hallucination evaluation (VHE). This paper contributes a ChatGPT-Prompted visual hall…
Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
Zhongzhi Chen, Xingwu Sun, Xianfeng Jiao +4
Despite the great success of large language models (LLMs) in various tasks, they suffer from generating hallucinations. We introduce Truth Forest, a method that enhances truthfulne…