687 citations · 1.7k across the 81 of their papers we have counts for
17 papers · 1 filter
Context-DPO: Aligning Language Models for Context-Faithfulness
Baolong Bi, Shaohan Huang, Yiwei Wang +11
Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intenti…
Multimodal Latent Language Modeling with Next-Token Diffusion
Yutao Sun, Hangbo Bao, Wenhui Wang +5
Multimodal generative models require a unified approach to handle both discrete data (e.g., text and code) and continuous data (e.g., image, audio, video). In this work, we propose…
RedStone: Curating General, Code, Math, and QA Data for Large Language Models
Yaoyao Chang, Lei Cui, Li Dong +13
Pre-training Large Language Models (LLMs) on high-quality, meticulously curated datasets is widely recognized as critical for enhancing their performance and generalization capabil…
MH-MoE: Multi-Head Mixture-of-Experts
Shaohan Huang, Xun Wu, Shuming Ma +1
Multi-Head Mixture-of-Experts (MH-MoE) demonstrates superior performance by using the multi-head mechanism to collectively attend to information from various representation spaces…
Textual Aesthetics in Large Language Models
Lingjie Jiang, Shaohan Huang, Xun Wu +1
Image aesthetics is a crucial metric in the field of image generation. However, textual aesthetics has not been sufficiently explored. With the widespread application of large lang…
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Ting Jiang, Shaohan Huang, Shengyue Luo +8
Low-rank adaptation is a popular parameter-efficient fine-tuning method for large language models. In this paper, we analyze the impact of low-rank updating, as implemented in LoRA…