5 papers · 1 filter
How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL
Qifu Wen, Shuaijun Liu, Zihan Zhou +2
Accurate posterior prediction need not require accurate approximation of Bayesian updates. We prove that an unbounded gap between the update maps can coexist with vanishing predict…
Echo-LoRA: Parameter-Efficient Fine-Tuning via Cross-Layer Representation Injection
Yihang Peng, Peng Jin, Jie Gong +4
Parameter-efficient fine-tuning (PEFT) has become a practical route for adapting large language models to downstream tasks, with LoRA-style methods being particularly attractive be…
GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stopping
Qifu Wen, Xi Zeng, Zihan Zhou +4
Early stopping monitors global validation loss and halts all parameter updates simultaneously, which is computationally costly for large transformers due to the extended time requi…
Discovering Hidden Algebraic Structures via Transformers with Rank-Aware Beam GRPO
Jaeha Lee, Gio Huh, Ning Su +1
Recent efforts have extended the capabilities of transformers in logical reasoning and symbolic computations. In this work, we investigate their capacity for non-linear latent patt…
Calibre: Towards Fair and Accurate Personalized Federated Learning with Self-Supervised Learning
Sijia Chen, Ningxin Su, Baochun Li
In the context of personalized federated learning, existing approaches train a global model to extract transferable representations, based on which any client could train personali…