9 citations · 12 across the 9 of their papers we have counts for
9 papers
LaMoC: Loss-Aware Modular Compression for LLMs
Mohanad Odema, Jacob Song
Modular compression has enabled considerable parameter reduction in LLMs while preserving strong language understanding and downstream task accuracy. However, existing joint modula…
Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs
Mohanad Odema, Gabrielle De Micheli, Dayin Gou +3
Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-le…
3-Model Speculative Decoding
Sanghyun Byun, Mohanad Odema, Jung Ick Guack +3
Speculative Decoding (SD) accelerates inference in large language models by using a smaller draft model to propose tokens, which are then verified by a larger target model. However…
Unifying Vision-Language Latents for Zero-label Image Caption Enhancement
Sanghyun Byun, Jung Ick Guack, Mohanad Odema +3
Vision-language models (VLMs) achieve remarkable performance through large-scale image-text pretraining. However, their reliance on labeled image datasets limits scalability and le…
CARVQ: Corrective Adaptor with Group Residual Vector Quantization for LLM Embedding Compression
Dayin Gou, Sanghyun Byun, Nilesh Malpeddi +4
Large Language Models (LLMs) typically rely on a large number of parameters for token embedding, leading to substantial storage requirements and memory footprints. In particular, L…
APCE: Adaptive Progressive Context Expansion for Long Context Processing
Baisub Lee, Sanghyun Byun, Mohanad Odema +3
Deploying useful Long-Context Transformer Models (LCTMs) requires addressing two key challenges: (1) A growing memory footprint due to quadratic self-attention and linear KV-cache…
HunyuanVideo: A Systematic Framework For Large Video Generative Models
Weijie Kong, Qi Tian, Zijian Zhang +49
Recent advancements in video generation have significantly impacted daily life for both individuals and industries. However, the leading video generation models remain closed-sourc…
OneNet: A Channel-Wise 1D Convolutional U-Net
Sanghyun Byun, Kayvan Shah, Ayushi Gang +3
Many state-of-the-art computer vision architectures leverage U-Net for its adaptability and efficient feature extraction. However, the multi-resolution convolutional design often l…
MultiDepth: Multi-Sample Priors for Refining Monocular Metric Depth Estimations in Indoor Scenes
Sanghyun Byun, Jacob Song, Woo Seong Chung
Monocular metric depth estimation (MMDE) is a crucial task to solve for indoor scene reconstruction on edge devices. Despite this importance, existing models are sensitive to facto…