4 papers
Parameter Reduction Improves Vision Transformers: A Comparative Study of Sharing and Width Reduction
Anantha Padmanaban Krishna Kumar
Although scaling laws and many empirical results suggest that increasing the size of Vision Transformers often improves performance, model accuracy and training behavior are not al…
Mechanisms of Non-Monotonic Scaling in Vision Transformers
Anantha Padmanaban Krishna Kumar
Deeper Vision Transformers often perform worse than shallower ones, which challenges common scaling assumptions. Through a systematic empirical analysis of ViT-S, ViT-B, and ViT-L…
Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels
Anantha Padmanaban Krishna Kumar
Can in-context learning (ICL) override pre-trained label semantics, or does it merely refine an existing semantic backbone? We address this question by treating LLMs as prompt-indu…
Attentions Under the Microscope: A Comparative Study of Resource Utilization for Variants of Self-Attention
Zhengyu Tian, Anantha Padmanaban Krishna Kumar, Hemant Krishnakumar +1
As large language models (LLMs) and visual language models (VLMs) grow in scale and application, attention mechanisms have become a central computational bottleneck due to their hi…