39 citations · 119 across the 12 of their papers we have counts for
12 papers
How Do In-Context Examples Affect Compositional Generalization?
Shengnan An, Zeqi Lin, Qiang Fu +4
Compositional generalization--understanding unseen combinations of seen primitives--is an essential reasoning capability in human intelligence. The AI community mainly studies this…
Skill-Based Few-Shot Selection for In-Context Learning
Shengnan An, Bo Zhou, Zeqi Lin +5
In-context learning is the paradigm that adapts large language models to downstream tasks by providing a few examples. Few-shot selection -- selecting appropriate examples for each…
Homophily-oriented Heterogeneous Graph Rewiring
Jiayan Guo, Lun Du, Wendong Bi +6
With the rapid development of the World Wide Web (WWW), heterogeneous graphs (HG) have explosive growth. Recently, heterogeneous graph neural network (HGNN) has shown great potenti…
Does Deep Learning Learn to Abstract? A Systematic Probing Framework
Shengnan An, Zeqi Lin, Bei Chen +3
Abstraction is a desirable capability for deep learning models, which means to induce abstract concepts from concrete instances and flexibly apply them beyond the learning context.…
Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach
Wendong Bi, Lun Du, Qiang Fu +3
Graph Neural Networks (GNNs) are popular machine learning methods for modeling graph data. A lot of GNNs perform well on homophily graphs while having unsatisfactory performance on…
Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima
Hengyu Liu, Qiang Fu, Lun Du +4
Learning rate is one of the most important hyper-parameters that has a significant influence on neural network training. Learning rate schedules are widely used in real practice to…
Making Large Language Models Better Reasoners with Step-Aware Verifier
Yifei Li, Zeqi Lin, Shizhuo Zhang +4
Few-shot learning is a challenging task that requires language models to generalize from limited examples. Large language models like GPT-3 and PaLM have made impressive progress i…
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models
Shengnan An, Yifei Li, Zeqi Lin +6
Recently the prompt-tuning paradigm has attracted significant attention. By only tuning continuous prompts with a frozen pre-trained language model (PLM), prompt-tuning takes a ste…
Reasoning Like Program Executors
Xinyu Pi, Qian Liu, Bei Chen +6
Reasoning over natural language is a long-standing goal for the research community. However, studies have shown that existing language models are inadequate in reasoning. To addres…
Neuron with Steady Response Leads to Better Generalization
Qiang Fu, Lun Du, Haitao Mao +4
Regularization can mitigate the generalization gap between training and inference by introducing inductive bias. Existing works have already proposed various inductive biases from…
GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily
Lun Du, Xiaozhou Shi, Qiang Fu +4
Graph Neural Networks (GNNs) are widely used on a variety of graph-based machine learning tasks. For node-level tasks, GNNs have strong power to model the homophily property of gra…
Neuron Campaign for Initialization Guided by Information Bottleneck Theory
Haitao Mao, Xu Chen, Qiang Fu +3
Initialization plays a critical role in the training of deep neural networks (DNN). Existing initialization strategies mainly focus on stabilizing the training process to mitigate…