4 papers
The Importance of Encoder Choice:A Tabular-Image Study
Ilia Koloiarov, Diego Coello de Portugal Mecke, Vijaya Krishna Yalavarthi +2
Multimodal learning usually requires a dedicated encoder per modality. When a tabular modality is involved, prior work has been mostly using a \emph{plain MLP} as the encoder. Yet…
Prune, Update and Trim: Robust Structured Pruning for Large Language Models
Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidt-Thieme
Large Language Models (LLMs) have experienced significant growth and development in recent years. However, performing inference on LLMs remains costly, especially for long-context…
HexFormer: Hyperbolic Vision Transformer with Exponential Map Aggregation
Haya Alyoussef, Ahmad Bdeir, Diego Coello de Portugal Mecke +3
Data across modalities such as images, text, and graphs often contains hierarchical and relational structures, which are challenging to model within Euclidean geometry. Hyperbolic…
STADE: Standard Deviation as a Pruning Metric
Diego Coello de Portugal Mecke, Haya Alyoussef, Maximilian Stubbemann +3
Recently, Large Language Models (LLMs) have become very widespread and are used to solve a wide variety of tasks. To successfully handle these tasks, LLMs require longer training t…