6 papers
KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning
Yinyi Luo, Zhexian Zhou, Hao Chen +4
Knowledge editing and machine unlearning are two popular approaches for large language models (LLMs) to stay up-to-date. However, the knowledge updating mechanism of LLMs remains l…
The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution
Frank Sifei Luan, Ron Yifeng Wang, Yile Gu +12
While ML model training and inference are both GPU-intensive, CPU-based data processing is often the bottleneck. Distributed data processing systems based on the batch or stream pr…
Turning Internal Gap into Self-Improvement: Promoting the Generation-Understanding Unification in MLLMs
Yujin Han, Hao Chen, Andi Han +5
Although unified MLLMs aim to unify generation and understanding, they are considered to exhibit an internal gap, with understanding outperforming generation. Through large-scale e…
Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics
Shiwei Li, Xiandi Luo, Xing Tang +6
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method. In standard LoRA layers, one of the matrices, or , is initialized to zero, ensuring that…
Inference performance evaluation for LLMs on edge devices with a novel benchmarking framework and metric
Hao Chen, Cong Tian, Zixuan He +3
With the significant success achieved by large language models (LLMs) like LLaMA, edge computing-based LLM inference services for mobile and PC are in high demand for data privacy.…
Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
Tai An, Ruwu Cai, Yanzhe Zhang +6
In the era of large language models (LLMs), N:M sparsity has emerged as a structured compression technique critical for accelerating inference. While prior work has primarily focus…