6 citations · 19 across the 27 of their papers we have counts for
31 papers
Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
Aojun Lu, Tao Feng, Hangjie Yuan +2
The adaptation of large-scale Vision-Language Models (VLMs) through post-training reveals a pronounced generalization gap: models fine-tuned with Reinforcement Learning (RL) consis…
C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning
Wei Li, Hangjie Yuan, Zixiang Zhao +4
Balancing sensitivity to new tasks and stability for retaining past knowledge is crucial in continual learning (CL). Recently, sharpness-aware minimization has proven effective in…
Surrogate-Assisted Evolution for Efficient Multi-branch Connection Design in Deep Neural Networks
Fergal Stapleton, Daniel García Núñez, Yanan Sun +1
State-of-the-art Deep Neural Networks (DNNs) often incorporate multi-branch connections, enabling multi-scale feature extraction and enhancing the capture of diverse features. This…
Runtime Analysis of Evolutionary NAS for Multiclass Classification
Zeqiong Lv, Chao Qian, Yun Liu +2
Evolutionary neural architecture search (ENAS) is a key part of evolutionary machine learning, which commonly utilizes evolutionary algorithms (EAs) to automatically design high-pe…
Loss Functions for Predictor-based Neural Architecture Search
Han Ji, Yuqi Feng, Jiahao Fan +1
Evaluation is a critical but costly procedure in neural architecture search (NAS). Performance predictors have been widely adopted to reduce evaluation costs by directly estimating…
Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective
Aojun Lu, Hangjie Yuan, Tao Feng +1
The quest for Continual Learning (CL) seeks to empower neural networks with the ability to learn and adapt incrementally. Central to this pursuit is addressing the stability-plasti…
CARL: Causality-guided Architecture Representation Learning for an Interpretable Performance Predictor
Han Ji, Yuqi Feng, Jiahao Fan +1
Performance predictors have emerged as a promising method to accelerate the evaluation stage of neural architecture search (NAS). These predictors estimate the performance of unsee…
Adapt before Continual Learning
Aojun Lu, Tao Feng, Hangjie Yuan +2
Continual Learning (CL) seeks to enable neural networks to incrementally acquire new knowledge (plasticity) while retaining existing knowledge (stability). Although pre-trained mod…
LAS: Loss-less ANN-SNN Conversion for Fully Spike-Driven Large Language Models
Long Chen, Xiaotian Song, Yanan Sun
Spiking Large Language Models (LLMs) have emerged as an energy-efficient alternative to conventional LLMs through their event-driven computation. To effectively obtain spiking LLMs…
E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing
Yuhao Zhou, Yuxin Tian, Mingjia Shi +4
The exponential growth in model sizes has significantly increased the communication burden in Federated Learning (FL). Existing methods to alleviate this burden by transmitting com…
Achieving Deep Continual Learning via Evolution
Aojun Lu, Junchao Ke, Chunhui Ding +3
Deep neural networks, despite their remarkable success, remain fundamentally limited in their ability to perform Continual Learning (CL). While most current methods aim to enhance…
FAS: Fast ANN-SNN Conversion for Spiking Large Language Models
Long Chen, Xiaotian Song, Andy Song +3
Spiking Large Language Models have been shown as a good alternative to LLMs in various scenarios. Existing methods for creating Spiking LLMs, i.e., direct training and ANN-SNN conv…
Revisiting Long-Tailed Learning: Insights from an Architectural Perspective
Yuhan Pan, Yanan Sun, Wei Gong
Long-Tailed (LT) recognition has been widely studied to tackle the challenge of imbalanced data distributions in real-world applications. However, the design of neural architecture…
CAP: A Context-Aware Neural Predictor for NAS
Han Ji, Yuqi Feng, Yanan Sun
Neural predictors are effective in boosting the time-consuming performance evaluation stage in neural architecture search (NAS), owing to their direct estimation of unseen architec…
Towards Accurate and Robust Architectures via Neural Architecture Search
Yuwei Ou, Yuqi Feng, Yanan Sun
To defend deep neural networks from adversarial attacks, adversarial training has been drawing increasing attention for its effectiveness. However, the accuracy and robustness resu…
Revisiting Neural Networks for Continual Learning: An Architectural Perspective
Aojun Lu, Tao Feng, Hangjie Yuan +2
Efforts to overcome catastrophic forgetting have primarily centered around developing more effective Continual Learning (CL) methods. In contrast, less attention was devoted to ana…
Genetic Auto-prompt Learning for Pre-trained Code Intelligence Language Models
Chengzhe Feng, Yanan Sun, Ke Li +3
As Pre-trained Language Models (PLMs), a popular approach for code intelligence, continue to grow in size, the computational cost of their usage has become prohibitively expensive.…
A First Step Towards Runtime Analysis of Evolutionary Neural Architecture Search
Zeqiong Lv, Chao Qian, Yanan Sun
Evolutionary neural architecture search (ENAS) employs evolutionary algorithms to find high-performing neural architectures automatically, and has achieved great success. However,…
Exploring the Robustness of Decentralized Training for Large Language Models
Lin Lu, Chenxi Dai, Wangcheng Tao +3
Decentralized training of large language models has emerged as an effective way to democratize this technology. However, the potential threats associated with this approach have no…
Differentiable Genetic Programming for High-dimensional Symbolic Regression
Peng Zeng, Xiaotian Song, Andrew Lensen +4
Symbolic regression (SR) is the process of discovering hidden relationships from data with mathematical expressions, which is considered an effective way to reach interpretable mac…
Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence
Yuhao Zhou, Mingjia Shi, Yuanxi Li +3
Reducing communication overhead in federated learning (FL) is challenging but crucial for large-scale distributed privacy-preserving machine learning. While methods utilizing spars…
Differentiable Search of Accurate and Robust Architectures
Yuwei Ou, Xiangning Xie, Shangce Gao +3
Deep neural networks (DNNs) are found to be vulnerable to adversarial attacks, and various methods have been proposed for the defense. Among these methods, adversarial training has…
DAS: Neural Architecture Search via Distinguishing Activation Score
Yuqiao Liu, Haipeng Li, Yanan Sun +1
Neural Architecture Search (NAS) is an automatic technique that can search for well-performed architectures for a specific task. Although NAS surpasses human-designed architecture…
Architecture Augmentation for Performance Predictor Based on Graph Isomorphism
Xiangning Xie, Yuqiao Liu, Yanan Sun +2
Neural Architecture Search (NAS) can automatically design architectures for deep neural networks (DNNs) and has become one of the hottest research topics in the current machine lea…