11 papers
Nonlinearity-Aware LoRA: Structured Gate Adaptation under Low-Rank Constraints
Shuai Yuan, Sudong Cai, Bingzhi Chen +4
Low-rank adaptation (LoRA) is commonly viewed as an update-space approximation to full fine-tuning, yet this view is incomplete for self-gated Transformer feed-forward networks. In…
ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws
Xuyang Cao, Qianying Liu, Chuan Xiao +7
In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the \textit{language mixtu…
Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
Rei Taniguchi, Yuyang Dong, Makoto Onizuka +1
Due to the prevalence of large language models (LLMs), key-value (KV) cache reduction for LLM inference has received remarkable attention. Among numerous works that have been propo…
GraphCompliance: Aligning Policy and Context Graphs for LLM-Based Regulatory Compliance
Jiseong Chung, Ronny Ko, Wonchul Yoo +4
Compliance at web scale poses practical challenges: each request may require a regulatory assessment. Regulatory texts (e.g., the General Data Protection Regulation, GDPR) are cros…
Seven Security Challenges in Cross-domain Multi-agent LLM Systems
Ronny Ko, Jiseong Jeong, Shuyuan Zheng +4
Large language models (LLMs) are rapidly evolving into autonomous agents that cooperate across organizational boundaries, enabling joint disaster response, supply-chain optimizatio…
CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics
Kuntian Zhang, Simin Yu, Yaoshu Wang +2
In this paper, we propose CKGAN, a novel generative adversarial network (GAN) variant based on an integral probability metrics framework with characteristic kernel (CKIPM). CKIPM,…