18 citations · 18 across the 6 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.SE2023★ 18 cited
CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model
Peng Di, Jianguo Li, Hang Yu +35
Code Large Language Models (Code LLMs) have gained significant attention in the industry due to their wide applications in the full lifecycle of software engineering. However, the…
cs.LG2023
CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
Shiyi Zhu, Jing Ye, Wei Jiang +4
Self-attention and position embedding are two key modules in transformer-based Large Language Models (LLMs). However, the potential relationship between them is far from well studi…