4 papers
Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory
Kaustubh Kapil, Kishor P. Upla
Transformers have had a profound impact on the world of language processing and computer vision. As efforts to answer the million-dollar question of ``How does a Transformer learn?…
Transformer Geometry Observatory TGO-III: Semantic Geometry Observatory
Kaustubh Kapil, Kishor P. Upla
With the widespread adoption of Vision Transformers in modern AI, the need to analyze their inherent representational behavior has become increasingly important. While most existin…
Transformer Geometry Observatory TGO-II: Representational Similarity Observatory
Kaustubh Kapil, Kishor P. Upla
While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout tr…
Transformer Geometry Observatory TGO-I: Spectral Geometry Observatory
Kaustubh Kapil, Kishor P. Upla
Despite the widespread adoption of Vision Transformers (ViTs) and their success across numerous computer vision applications, the fundamental understanding of their dimensional and…