72 citations · 330 across the 48 of their papers we have counts for
7 papers · 1 filter
When Less Data Favors Smaller Teachers: Rethinking Teacher Capacity and Data Selection for Knowledge Distillation
Minjae Park, Taesun Yeom, Jaeho Lee
Data pruning reduces the training cost of knowledge distillation (KD). However, the preferred teacher capacity changes with the data budget: smaller teachers can outperform larger…
KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation
Sihyeon Ha, Jaeho Lee, Yo-Seb Jeon
What limits KV-cache compression at extreme bit-rates? We argue that it is not the choice of compression scheme, but how its budget is allocated across attention heads. Existing me…
Token-Oriented Semantic Communication with Pretrained Vision Transformers
Jiwoong Im, Minwoo Kim, Jaeho Lee +2
Token communications realize the semantic communication principle at the granularity of transformer tokens, providing a promising direction for client--server collaborative inferen…
Learned Image Compression for Vision-Language-Action Models
Hyeonjun Kim, Jegwang Ryu, Sangbeom Ha +4
Vision-language-action (VLA) models increasingly rely on high-frequency multi-camera observations, making visual communication a major bottleneck for real-time robotic control in b…
Multi-frame Restoration for 10 Hz Lissajous Confocal Laser Endomicroscopy
Minhee Lee, Sangyoon Lee, Jiwook Lee +4
Lissajous confocal laser endomicroscopy (CLE) is a promising solution for high-speed in vivo optical biopsy for handheld scenarios. However, Lissajous scanning traces a resonant tr…
Over-Alignment vs Over-Fitting: The Role of Feature Learning Strength in Generalization
Taesun Yeom, Taehyeok Ha, Jaeho Lee
Feature learning strength (FLS), i.e., the inverse of the effective output scaling of a model, plays a critical role in shaping the optimization dynamics of neural nets. While its…
Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models
Hyunjong Ok, Jaeho Lee
Large language models exhibit surprising sensitivity to the structure of the prompt, but the mechanisms underlying this sensitivity remain poorly understood. In this work, we condu…