9 citations · 11 across the 4 of their papers we have counts for
6 papers
LeRoPE: Learnable RoPE Frequencies Improve Language Modeling
Petros Karypis, Sean O'Brien, Shreyas Kadekodi +2
Rotary Positional Encodings (RoPE) are currently the most popular positional encodings used in modern language models. RoPE rotates two-dimensional chunks of query and key vectors,…
Fine-Tuning Language Models on Multiple Datasets for Citation Intention Classification
Zeren Shui, Petros Karypis, Daniel S. Karls +4
Citation intention Classification (CIC) tools classify citations by their intention (e.g., background, motivation) and assist readers in evaluating the contribution of scientific l…
Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
Costas Mavromatis, Petros Karypis, George Karypis
Fusing knowledge from multiple Large Language Models (LLMs) can combine their diverse strengths to achieve improved performance on a given task. However, current fusion approaches…
SemPool: Simple, robust, and interpretable KG pooling for enhancing language models
Costas Mavromatis, Petros Karypis, George Karypis
Knowledge Graph (KG) powered question answering (QA) performs complex reasoning over language semantics as well as knowledge facts. Graph Neural Networks (GNNs) learn to aggregate…
Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models
An Yan, Yu Wang, Yiwu Zhong +8
Medical image classification is a critical problem for healthcare, with the potential to alleviate the workload of doctors and facilitate diagnoses of patients. However, two challe…
Extending Input Contexts of Language Models through Training on Segmented Sequences
Petros Karypis, Julian McAuley, George Karypis
Effectively training language models on long inputs poses many technical challenges. As a cost consideration, languages models are pretrained on a fixed sequence length before bein…