5 papers
POT: Inducing Overthinking in LLMs via Black-Box Iterative Optimization
Xinyu Li, Tianjin Huang, Ronghui Mu +2
Recent advances in Chain-of-Thought (CoT) prompting have substantially enhanced the reasoning capabilities of large language models (LLMs), enabling sophisticated problem-solving t…
LOST: Low-rank and Sparse Pre-training for Large Language Models
Jiaxi Li, Lu Yin, Li Shen +6
While large language models (LLMs) have achieved remarkable performance across a wide range of tasks, their massive scale incurs prohibitive computational and memory costs for pre-…
Principal Eigenvalue Regularization for Improved Worst-Class Certified Robustness of Smoothed Classifiers
Gaojie Jin, Tianjin Huang, Ronghui Mu +1
Recent studies have identified a critical challenge in deep neural networks (DNNs) known as ``robust fairness", where models exhibit significant disparities in robust accuracy acro…
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Tianjin Huang, Ziquan Zhu, Gaojie Jin +3
Large Language Models (LLMs) have demonstrated exceptional performance across diverse tasks, yet their training remains highly resource-intensive and susceptible to critical challe…
Enhancing Robust Fairness via Confusional Spectral Regularization
Gaojie Jin, Sihao Wu, Jiaxu Liu +2
Recent research has highlighted a critical issue known as ``robust fairness", where robust accuracy varies significantly across different classes, undermining the reliability of de…