12 papers
PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
Zhengtao Yao, Runhao Li, Xupeng Chen +12
Discrete masked diffusion language models support bidirectional generation and infilling, but adapting pretrained autoregressive (AR) transformers requires reconciling causal pretr…
Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization
Zhengtao Yao, Runhao Li, Xupeng Chen +12
Research on preference optimization often varies the training objective while holding the data fixed. We instead ask whether a small, high-confidence set of on-policy responses can…
Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
Yujia Tong, Yuxi Wang, Yunyang Wan +3
Model compression techniques such as quantization and pruning are widely used to reduce the deployment cost of large language models (LLMs), with existing evaluations focusing almo…
TSNN: A Non-parametric and Interpretable Framework for Traffic Time Series Forecasting
Bowen Liu, Haijian Lai, Chan-Tong Lam +4
Although many complex models were proposed to analyze time series data, some studies have demonstrated remarkable performance with simpler structures. A recent study proposed a non…
SAU: Sparsity-Aware Unlearning for LLMs via Gradient Masking and Importance Redistribution
Yuze Wang, Yujia Tong, Xuan Liu +1
Large Language Models (LLMs) inevitably memorize sensitive information during training, posing significant privacy risks. Machine unlearning has emerged as a promising solution to…
ChromouVQA: Benchmarking Vision-Language Models under Chromatic Camouflaged Images
Yunfei Zhang, Yizhuo He, Yuanxun Shao +5
Vision-Language Models (VLMs) have advanced multimodal understanding, yet still struggle when targets are embedded in cluttered backgrounds requiring figure-ground segregation. To…
A Systematic Study of Model Extraction Attacks on Graph Foundation Models
Haoyan Xu, Ruizhi Qian, Jiate Li +9
Graph machine learning has advanced rapidly in tasks such as link prediction, anomaly detection, and node classification. As models scale up, pretrained graph models have become va…
JEPA-T: Joint-Embedding Predictive Architecture with Text Fusion for Image Generation
Siheng Wan, Zhengtao Yao, Zhengdao Li +9
Modern Text-to-Image (T2I) generation increasingly relies on token-centric architectures that are trained with self-supervision, yet effectively fusing text with visual tokens rema…
OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment
Liang Lin, Zhihao Xu, Junhao Dong +10
Large language model (LLM) alignment faces a critical dilemma when addressing multiple human preferences: improvements in one dimension frequently come at the expense of others, cr…
AMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models
Yuqi Li, Chuanguang Yang, Junhao Dong +6
The success of large-scale visual language pretraining (VLP) models has driven widespread adoption of image-text retrieval tasks. However, their deployment on mobile devices remain…
SRKD: Towards Efficient 3D Point Cloud Segmentation via Structure- and Relation-aware Knowledge Distillation
Yuqi Li, Junhao Dong, Zeyu Dong +3
3D point cloud segmentation faces practical challenges due to the computational complexity and deployment limitations of large-scale transformer-based models. To address this, we p…
SepPrune: Structured Pruning for Efficient Deep Speech Separation
Yuqi Li, Kai Li, Xin Yin +6
Although deep learning has substantially advanced speech separation in recent years, most existing studies continue to prioritize separation quality while overlooking computational…