5 papers
Vertical Fusion: Condensing Internal Representations for Robust ViT Classification
Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi +4
Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is conside…
Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
Hamed Damirchi, Ignacio Meza De la Jara, Ehsan Abbasnejad +3
Existing explainability methods for Large Language Models (LLMs) typically treat hidden states as static points in activation space, assuming that correct and incorrect inferences…
An empirical study of the effect of video encoders on Temporal Video Grounding
Ignacio M. De la Jara, Cristian Rodriguez-Opazo, Edison Marrese-Taylor +1
Temporal video grounding is a fundamental task in computer vision, aiming to localize a natural language query in a long, untrimmed video. It has a key role in the scientific commu…
Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection
I. M. De la Jara, C. Rodriguez-Opazo, D. Teney +2
Out-of-distribution (OOD) detection is essential for reliably deploying machine learning models in the wild. Yet, most methods treat large pre-trained models as monolithic encoders…
Unveiling Backbone Effects in CLIP: Exploring Representational Synergies and Variances
Cristian Rodriguez-Opazo, Edison Marrese-Taylor, Ehsan Abbasnejad +4
Contrastive Language-Image Pretraining (CLIP) stands out as a prominent method for image representation learning. Various neural architectures, spanning Transformer-based models li…