7 papers
Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling
Changze Lv, Zhenghua Wang, Yiran Ding +9
Large Language Models (LLMs) still struggle with the ``lost-in-the-middle'' problem, where critical information located in the middle of long-context inputs is often underrepresent…
BatCoder: Self-Supervised Bidirectional Code-Documentation Learning via Back-Translation
Jingwen Xu, Yiyang Lu, Zisu Huang +9
Training LLMs for code-related tasks typically depends on high-quality code-documentation pairs, which are costly to curate and often scarce for niche programming languages. We int…
Enhancing Model Privacy in Federated Learning with Random Masking and Quantization
Zhibo Xu, Jianhao Zhu, Jingwen Xu +7
The primary goal of traditional federated learning is to protect data privacy by enabling distributed edge devices to collaboratively train a shared global model while keeping raw…
Enhancing the Capability and Robustness of Large Language Models through Reinforcement Learning-Driven Query Refinement
Xiaohua Wang, Zisu Huang, Feiran Zhang +5
The capacity of large language models (LLMs) to generate honest, harmless, and helpful responses heavily relies on the quality of user prompts. However, these prompts often tend to…
Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning
Muling Wu, Qi Qian, Wenhao Liu +12
Large Language Models (LLMs) have achieved remarkable performance across various reasoning tasks, yet post-training is constrained by inefficient sample utilization and inflexible…
Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Changze Lv, Jingwen Xu, Yiyang Lu +7
Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's success. However, its biological plausibility has been challenged due…