8 papers
Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
Kexuan Shi, Hanxuan Li, Zeju Qiu +3
We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam a…
Orthogonal Model Merging
Sihan Yang, Kexuan Shi, Weiyang Liu
Merging finetuned Large Language Models (LLMs) has become increasingly important for integrating diverse capabilities into a single unified model. However, prevailing model merging…
Consistency Trajectory Matching for One-Step Generative Super-Resolution
Weiyi You, Mingyang Zhang, Leheng Zhang +3
Current diffusion-based super-resolution (SR) approaches achieve commendable performance at the cost of high inference overhead. Therefore, distillation techniques are utilized to…
Model Merging with Functional Dual Anchors
Kexuan Shi, Yandong Wen, Weiyang Liu
Model merging is an efficient post-training strategy for integrating knowledge from multiple finetuned checkpoints of a shared foundation model. Existing methods operate in the par…
Task-Aware Image Signal Processor for Advanced Visual Perception
Kai Chen, Jin Xiao, Leheng Zhang +2
In recent years, there has been a growing trend in computer vision towards exploiting RAW sensor data, which preserves richer information compared to conventional low-bit RGB image…
Learning Pixel-adaptive Multi-layer Perceptrons for Real-time Image Enhancement
Junyu Lou, Xiaorui Zhao, Kexuan Shi +1
Deep learning-based bilateral grid processing has emerged as a promising solution for image enhancement, inherently encoding spatial and intensity information while enabling effici…