4 papers
UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization
Longfei Yun, Letian Peng, Jingbo Shang
Fine granularity is an essential requirement for controllable text generation, which has seen rapid growth with the ability of LLMs. However, existing methods focus mainly on a sma…
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's Nest
Letian Peng, Zilong Wang, Feng Yao +1
Massive high-quality data, both pre-training raw texts and post-training annotations, have been carefully prepared to incubate advanced large language models (LLMs). In contrast, f…
Linear Correlation in LM's Compositional Generalization and Hallucination
Letian Peng, Chenyang An, Shibo Hao +2
The generalization of language models (LMs) is undergoing active debates, contrasting their potential for general intelligence with their struggles with basic knowledge composition…
Correlation and Navigation in the Vocabulary Key Representation Space of Language Models
Letian Peng, Chenyang An, Jingbo Shang
Language model (LM) decoding is based on the next-token prediction (NTP) probability distribution. For neural LMs (e.g., Transformer-based), NTP distribution is essentially a softm…