5 papers
CORAM: Coherent Orthogonal Rotation for Model Merging
Xinyi Sui, Ziran Liu, Nam Ling +2
Merging finetuned models combines specialized capabilities without joint training or access to the original data. Most methods operate by linear arithmetic in Euclidean weight spac…
CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation
Pengcheng Wang, Ziran Liu, Wei Wang +1
Parameter-Efficient Fine-Tuning (PEFT) commonly adapts pretrained weights through low-rank updates, and recent methods further exploit the singular value decomposition (SVD) of the…
Geometric and Spectral Alignment for Deep Neural Network II
Ziran Liu, Wei Wang, Jinhao Wang +5
This paper develops the angular and static-channel component of Geometric and Spectral Alignment for residual Jacobian chains. Starting from Cartan-coordinate rigidity and fitted e…
Geometric and Spectral Alignment for Deep Neural Network I
Ziran Liu, Wei Wang, Jinhao Wang +5
Deep residual architectures are modeled as products of near-identity Jacobians. This paper proves deterministic quotient-geometric estimates for singular spectra of Frobenius-norma…
Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning
Ziran Liu
Internal noise in deep networks is usually inherited from heuristics such as dropout, hard masking, or additive perturbation. We ask two questions: what correlation geometry should…