72 citations · 136 across the 18 of their papers we have counts for
18 papers
From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora
Yingli Shen, Wen Lai, Shuo Wang +4
Continued pretraining and instruction tuning on large-scale multilingual data have proven to be effective in scaling large language models (LLMs) to low-resource languages. However…
Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognition
Xiaoying Zhang, Da Peng, Yipeng Zhang +7
Recent progress in (multimodal) large language models ((M)LLMs) has shifted focus from pre-training to inference-time computation and post-training optimization, largely due to con…
KBAlign: Efficient Self Adaptation on Specific Knowledge Bases
Zheni Zeng, Yuxuan Chen, Shi Yu +7
Although retrieval-augmented generation (RAG) remains essential for knowledge-based question answering (KBQA), current paradigms face critical challenges under specific domains. Ex…
LLMMapReduce: Simplified Long-Sequence Processing using Large Language Models
Zihan Zhou, Chong Li, Xinyi Chen +11
Enlarging the context window of large language models (LLMs) has become a crucial research area, particularly for applications involving extremely long texts. In this work, we prop…
The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning
Bingxiang He, Ning Ding, Cheng Qian +10
Understanding alignment techniques begins with comprehending zero-shot generalization brought by instruction tuning, but little of the mechanism has been understood. Existing work…
Scaling Large Language Model-based Multi-Agent Collaboration
Chen Qian, Zihao Xie, YiFei Wang +9
Recent breakthroughs in large language model-driven autonomous agents have revealed that multi-agent collaboration often surpasses each individual through collective reasoning. Ins…
Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
Ao Sun, Weilin Zhao, Xu Han +5
The emergence of large language models (LLMs) relies heavily on distributed training strategies, among which pipeline parallelism plays a crucial role. As LLMs' training sequence l…
RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness
Tianyu Yu, Haoye Zhang, Qiming Li +13
Traditional feedback learning for hallucination reduction relies on labor-intensive manual labeling or expensive proprietary models. This leaves the community without foundational…
Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models
Ning Ding, Yulin Chen, Ganqu Cui +6
Underlying data distributions of natural language, programming code, and mathematical symbols vary vastly, presenting a complex challenge for large language models (LLMs) that stri…
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
Haoyu Wang, Shuo Wang, Yukun Yan +9
Open-source large language models (LLMs) have gained significant strength across diverse fields. Nevertheless, the majority of studies primarily concentrate on English, with only l…
Experiential Co-Learning of Software-Developing Agents
Chen Qian, Yufan Dang, Jiahao Li +9
Recent advancements in large language models (LLMs) have brought significant changes to various domains, especially through LLM-driven autonomous agents. A representative scenario…
Rational Decision-Making Agent with Internalized Utility Judgment
Yining Ye, Xin Cong, Shizuo Tian +5
Large language models (LLMs) have demonstrated remarkable advancements and have attracted significant efforts to develop LLMs into agents capable of executing intricate multi-step…
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Yujia Qin, Shihao Liang, Yining Ye +16
Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to…
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning
Yusheng Su, Chi-Min Chan, Jiali Cheng +9
Parameter-efficient tuning (PET) methods can effectively drive extremely large pre-trained language models (PLMs) by training only minimal parameters. Different PET methods utilize…
Chord-Conditioned Melody Harmonization with Controllable Harmonicity
Shangda Wu, Xiaobing Li, Maosong Sun
Melody harmonization has long been closely associated with chorales composed by Johann Sebastian Bach. Previous works rarely emphasised chorale generation conditioned on chord prog…
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
Fanchao Qi, Yangyi Chen, Mukai Li +3
Backdoor attacks are a kind of emergent training-time threat to deep neural networks (DNNs). They can manipulate the output of DNNs and possess high insidiousness. In the field of…
A Unified Framework for Community Detection and Network Representation Learning
Cunchao Tu, Xiangkai Zeng, Hao Wang +5
Network representation learning (NRL) aims to learn low-dimensional vectors for vertices in a network. Most existing NRL methods focus on learning representations from local contex…
Neural Headline Generation with Sentence-wise Optimization
Ayana, Shiqi Shen, Yu Zhao +2
Recently, neural models have been proposed for headline generation by learning to map documents to headlines with recurrent neural networks. Nevertheless, as traditional neural net…