Publications (23)
Revisiting Self-Supervised Heterogeneous Graph Learning from Spectral Clustering Perspective
Yujie Mo, Zhihe Lu, Runpeng Yu +2
Self-supervised heterogeneous graph learning (SHGL) has shown promising potential in diverse scenarios. However, while existing SHGL methods share a similar essential with clusteri…
OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization
Nanyang Ye, Kaican Li, Haoyue Bai +5
Deep learning has achieved tremendous success with independent and identically distributed (i.i.d.) data. However, the performance of neural networks often degenerates drastically…
HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters
Yujie Mo, Runpeng Yu, Xiaofeng Zhu +1
The "pre-train, prompt-tuning'' paradigm has demonstrated impressive performance for tuning pre-trained heterogeneous graph neural networks (HGNNs) by mitigating the gap between pr…
Through the Dual-Prism: A Spectral Perspective on Graph Data Augmentation for Graph Classification
Yutong Xia, Runpeng Yu, Yuxuan Liang +3
Graph Neural Networks have become the preferred tool to process graph data, with their efficacy being boosted through graph data augmentation techniques. Despite the evolution of a…
Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behavior
Bo Yin, Qi Li, Runpeng Yu +1
Instruction tuning increasingly relies on LLM-based prompt refinement, where prompts in the training corpus are selectively rewritten by an external refiner to improve clarity and…
KAN or MLP: A Fairer Comparison
Runpeng Yu, Weihao Yu, Xinchao Wang
This paper does not introduce a novel method. Instead, it offers a fairer and more comprehensive comparison of KAN and MLP models across various tasks, including machine learning,…
Introducing Visual Perception Token into Multimodal Large Language Model
Runpeng Yu, Xinyin Ma, Xinchao Wang
To utilize visual information, Multimodal Large Language Model (MLLM) relies on the perception process of its vision encoder. The completeness and accuracy of visual perception sig…
CoT-Valve: Length-Compressible Chain-of-Thought Tuning
Xinyin Ma, Guangnian Wan, Runpeng Yu +2
Chain-of-Thought significantly enhances a model's reasoning capability, but it also comes with a considerable increase in inference costs due to long chains. With the observation t…
Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs
Qi Li, Runpeng Yu, Haiquan Lu +1
Discrete Diffusion Large Language Models (dLLMs) have recently emerged as a competitive paradigm for non-autoregressive language modeling. Their distinctive decoding mechanism enab…
Multi-Level Collaboration in Model Merging
Qi Li, Runpeng Yu, Xinchao Wang
Parameter-level model merging is an emerging paradigm in multi-task learning with significant promise. Previous research has explored its connections with prediction-level model en…
Discrete Diffusion in Large Language and Multimodal Models: A Survey
Runpeng Yu, Qi Li, Xinchao Wang
In this work, we provide a systematic survey of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs). Unlike autoregressive (AR) mo…
Distribution Shift Inversion for Out-of-Distribution Prediction
Runpeng Yu, Songhua Liu, Xingyi Yang +1
Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the training and the testing…
Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
Qi Li, Bo Yin, Weiqi Huang +6
Vision-Language-Action (VLA) models are emerging as a unified substrate for embodied intelligence. This shift raises a new class of safety challenges, stemming from the embodied na…
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
Runpeng Yu, Xinyin Ma, Xinchao Wang
In this work, we propose Dimple, the first Discrete Diffusion Multimodal Large Language Model (DMLLM). We observe that training with a purely discrete diffusion approach leads to s…
Generator Born from Classifier
Runpeng Yu, Xinchao Wang
In this paper, we make a bold attempt toward an ambitious task: given a pre-trained classifier, we aim to reconstruct an image generator, without relying on any data samples. From…
dKV-Cache: The Cache for Diffusion Language Models
Xinyin Ma, Runpeng Yu, Gongfan Fang +1
Diffusion Language Models (DLMs) have been seen as a promising competitor for autoregressive language models. However, diffusion language models have long been constrained by slow…
Robust Long-Tailed Learning via Label-Aware Bounded CVaR
Hong Zhu, Runpeng Yu, Xing Tang +3
Data in the real-world classification problems are always imbalanced or long-tailed, wherein the majority classes have the most of the samples that dominate the model training. In…
NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
Lingfeng Ren, Weihao Yu, Runpeng Yu +1
Object hallucination is a critical issue in Large Vision-Language Models (LVLMs), where outputs include objects that do not appear in the input image. A natural question arises fro…
Neural Lineage
Runpeng Yu, Xinchao Wang
Given a well-behaved neural network, is possible to identify its parent, based on which it was tuned? In this paper, we introduce a novel task known as neural lineage detection, ai…
Attention Prompting on Image for Large Vision-Language Models
Runpeng Yu, Weihao Yu, Xinchao Wang
Compared with Large Language Models (LLMs), Large Vision-Language Models (LVLMs) can also accept images as input, thus showcasing more interesting emergent capabilities and demonst…
Encapsulating Knowledge in One Prompt
Qi Li, Runpeng Yu, Xinchao Wang
This paradigm encapsulates knowledge from various models into a solitary prompt without altering the original models or requiring access to the training data, which enables us to a…
Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms
Runpeng Yu, Hong Zhu, Kaican Li +5
Due to the poor generalization performance of traditional empirical risk minimization (ERM) in the case of distributional shift, Out-of-Distribution (OoD) generalization algorithms…
Vid-SME: Membership Inference Attacks against Large Video Understanding Models
Qi Li, Runpeng Yu, Xinchao Wang
Multimodal large language models (MLLMs) demonstrate remarkable capabilities in handling complex multimodal tasks and are increasingly adopted in video understanding applications.…