1 citations · 1 across the 11 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.AI2023
Transformer as Linear Expansion of Learngene
Shiyu Xia, Miaosen Zhang, Xu Yang +3
We propose expanding the shared Transformer module to produce and initialize Transformers of varying depths, enabling adaptation to diverse resource constraints. Drawing an analogy…
cs.LG2023
Building Variable-sized Models via Learngene Pool
Boyu Shi, Shiyu Xia, Xu Yang +3
Recently, Stitchable Neural Networks (SN-Net) is proposed to stitch some pre-trained networks for quickly building numerous networks with different complexity and performance trade…
cs.CV2023★ 1 cited
Manipulating the Label Space for In-Context Classification
Haokun Chen, Xu Yang, Yuhang Huang +3
After pre-training by generating the next word conditional on previous words, the Language Model (LM) acquires the ability of In-Context Learning (ICL) that can learn a new task co…