5 papers
Decoder-based Sense Knowledge Distillation
Qitong Wang, Mohammed J. Zaki, Georgios Kollias +1
Large language models (LLMs) learn contextual embeddings that capture rich semantic information, yet they often overlook structured lexical knowledge such as word senses and relati…
Multi-Sense Embeddings for Language Models and Knowledge Distillation
Qitong Wang, Mohammed J. Zaki, Georgios Kollias +1
Transformer-based large language models (LLMs) rely on contextual embeddings which generate different (continuous) representations for the same token depending on its surrounding c…
Can Memory-Augmented Language Models Generalize on Reasoning-in-a-Haystack Tasks?
Payel Das, Ching-Yun Ko, Sihui Dai +3
Large language models often expose their brittleness in reasoning tasks, especially while executing long chains of reasoning over context. We propose MemReasoner, a new and simple…
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan +6
Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a signif…
Large Language Models can be Strong Self-Detoxifiers
Ching-Yun Ko, Pin-Yu Chen, Payel Das +6
Reducing the likelihood of generating harmful and toxic output is an essential task when aligning large language models (LLMs). Existing methods mainly rely on training an external…