370 citations · 1.4k across the 49 of their papers we have counts for
12 papers · 1 filter
Qwen Technical Report
Jinze Bai, Shuai Bai, Yunfei Chu +45
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to hu…
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
Zheng Yuan, Hongyi Yuan, Chengpeng Li +5
Mathematical reasoning is a challenging task for large language models (LLMs), while the scaling relationship of it with respect to LLM capacity is under-explored. In this paper, w…
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning
Jianing Wang, Chengyu Wang, Chuanqi Tan +2
Large language models (LLMs) enable in-context learning (ICL) by conditioning on a few labeled training examples as a text-based prompt, eliminating the need for parameter updates…
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
Keming Lu, Hongyi Yuan, Zheng Yuan +5
Foundation language models obtain the instruction-following ability through supervised fine-tuning (SFT). Diversity and complexity are considered critical factors of a successful S…
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning
Zhen-Ru Zhang, Chuanqi Tan, Haiyang Xu +3
Fine-tuning large pre-trained language models on various downstream tasks with whole parameters is prohibitively expensive. Hence, Parameter-efficient fine-tuning has attracted att…
Knowledge Rumination for Pre-trained Language Models
Yunzhi Yao, Peng Wang, Shengyu Mao +4
Previous studies have revealed that vanilla pre-trained language models (PLMs) lack the capacity to handle knowledge-intensive NLP tasks alone; thus, several works have attempted t…
VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning
Zhen-Ru Zhang, Chuanqi Tan, Songfang Huang +1
Recent studies have demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages. In addition to involving the maske…
RRHF: Rank Responses to Align Language Models with Human Feedback without tears
Zheng Yuan, Hongyi Yuan, Chuanqi Tan +3
Reinforcement Learning from Human Feedback (RLHF) facilitates the alignment of large language models with human preferences, significantly enhancing the quality of interactions bet…
How well do Large Language Models perform in Arithmetic tasks?
Zheng Yuan, Hongyi Yuan, Chuanqi Tan +2
Large language models have emerged abilities including chain-of-thought to answer math word problems step by step. Solving math word problems not only requires abilities to disasse…
RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training
Zheng Yuan, Qiao Jin, Chuanqi Tan +4
Vision-and-language multi-modal pretraining and fine-tuning have shown great success in visual question answering (VQA). Compared to general domain VQA, the performance of biomedic…
Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling
Zheng Yuan, Yaoyun Zhang, Chuanqi Tan +3
Molecular dynamic simulations are important in computational physics, chemistry, material, and biology. Machine learning-based methods have shown strong abilities in predicting mol…
One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NER
Xiang Chen, Lei Li, Shuofei Qiao +5
Cross-domain NER is a challenging task to address the low-resource problem in practical scenarios. Previous typical solutions mainly obtain a NER model by pre-trained language mode…