5 papers
Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs
Mohanad Odema, Gabrielle De Micheli, Dayin Gou +3
Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-le…
3-Model Speculative Decoding
Sanghyun Byun, Mohanad Odema, Jung Ick Guack +3
Speculative Decoding (SD) accelerates inference in large language models by using a smaller draft model to propose tokens, which are then verified by a larger target model. However…
Unifying Vision-Language Latents for Zero-label Image Caption Enhancement
Sanghyun Byun, Jung Ick Guack, Mohanad Odema +3
Vision-language models (VLMs) achieve remarkable performance through large-scale image-text pretraining. However, their reliance on labeled image datasets limits scalability and le…
APCE: Adaptive Progressive Context Expansion for Long Context Processing
Baisub Lee, Sanghyun Byun, Mohanad Odema +3
Deploying useful Long-Context Transformer Models (LCTMs) requires addressing two key challenges: (1) A growing memory footprint due to quadratic self-attention and linear KV-cache…
Performance Implications of Multi-Chiplet Neural Processing Units on Autonomous Driving Perception
Mohanad Odema, Luke Chen, Hyoukjun Kwon +1
We study the application of emerging chiplet-based Neural Processing Units to accelerate vehicular AI perception workloads in constrained automotive settings. The motivation stems…