2 citations · 4 across the 7 of their papers we have counts for
7 papers
LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining
Qiuwu Chen, Zimo Liu, Yuchen Li +8
Large language models (LLMs) have achieved remarkable breakthroughs across various applications. However, their architectures remain inefficient in pretraining due to two main limi…
Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning
Kaiwen Chen, Shuhai Zhang, Zimo Liu +7
Large-scale neural network training increasingly relies on matrix-aware optimizers that exploit the structure of weight parameters beyond element-wise adaptation. However, existing…
Latent-Condensed Transformer for Efficient Long Context Modeling
Zeng You, Yaofo Chen, Qiuwu Chen +5
Large language models (LLMs) face significant challenges in processing long contexts due to the linear growth of the key-value (KV) cache and quadratic complexity of self-attention…
Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey
Junqiao Wang, Zeng Zhang, Yangfan He +18
With the rapid evolution of large language models (LLM), reinforcement learning (RL) has emerged as a pivotal technique for code generation and optimization in various domains. Thi…
Infant Agent: A Tool-Integrated, Logic-Driven Agent with Cost-Effective API Usage
Bin Lei, Yuchen Li, Yiming Zeng +7
Despite the impressive capabilities of large language models (LLMs), they currently exhibit two primary limitations, \textbf{\uppercase\expandafter{\romannumeral 1}}: They struggle…
FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization system
Zeyuan Li, Yangfan He, Lewei He +5
Recently, large language models (LLMs) have achieved significant progress in automated code generation. Despite their strong instruction-following capabilities, these models freque…
AutoCoder: Enhancing Code Large Language Model with \textsc{AIEV-Instruct}
Bin Lei, Yuchen Li, Qiuwu Chen
We introduce AutoCoder, the first Large Language Model to surpass GPT-4 Turbo (April 2024) and GPT-4o in pass@1 on the Human Eval benchmark test ( vs. $\mathbf{90.…