5 papers · 1 filter
Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
Mingyu Jin, Yutong Yin, Jingcheng Niu +7
In this work, we investigate how Large Language Models (LLMs) adapt their internal representations when encountering inputs of increasing difficulty, quantified as the degree of ou…
Baichuan-M1: Pushing the Medical Capability of Large Language Models
Bingning Wang, Haizhou Zhao, Huozhi Zhou +39
The current generation of large language models (LLMs) is typically designed for broad, general-purpose applications, while domain-specific LLMs, especially in vertical fields like…
KV Shifting Attention Enhances Language Modeling
Mingyu Xu, Wei Cheng, Bingning Wang +1
The current large language models are mainly based on decode-only structure transformers, which have great in-context learning (ICL) capabilities. It is generally believed that the…
Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adapt
Peter Anderson, Mano Vikash Janardhanan, Jason He +2
Financial documents are filled with specialized terminology, arcane jargon, and curious acronyms that pose challenges for general-purpose text embeddings. Yet, few text embeddings…
Calibrate to Discriminate: Improve In-Context Learning with Label-Free Comparative Inference
Wei Cheng, Tianlu Wang, Yanmin Ji +3
While in-context learning with large language models (LLMs) has shown impressive performance, we have discovered a unique miscalibration behavior where both correct and incorrect p…