5 papers · 1 filter
GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding
Ziyin Zhang, Hang Yu, Shijie Li +3
Programming languages possess rich semantic information - such as data flow - that is represented by graphs and not available from the surface form of source code. Recent code lang…
LAMDAS: LLM as an Implicit Classifier for Domain-specific Data Selection
Jian Wu, Hang Yu, Bingchang Liu +4
Adapting large language models (LLMs) to specific domains often faces a critical bottleneck: the scarcity of high-quality, human-curated data. While large volumes of unchecked data…
Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions
Zhihao He, Hang Yu, Zi Gong +3
Recent advancements in Transformer-based large language models (LLMs) have set new standards in natural language processing. However, the classical softmax attention incurs signifi…
CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models
Zi Gong, Hang Yu, Cong Liao +3
Multi-task learning (MTL) benefits the fine-tuning of large language models (LLMs) by providing a single model with improved performance and generalization ability across tasks, pr…
Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code
Ziyin Zhang, Chaoyu Chen, Bingchang Liu +5
In this work we systematically review the recent advancements in software engineering with language models, covering 70+ models, 40+ evaluation tasks, 180+ datasets, and 900 relate…