5 papers
Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
Junlin He, Yihong Tang, Tong Nie +5
Efficient Distillation (EDistill) compresses large language models (LLMs) by structured pruning parameters and tuning lightweight modules with high training efficiency. Although th…
From Sequential to Recursive: Enhancing Decision-Focused Learning with Bidirectional Feedback
Xinyu Wang, Jinxiao Du, Yiyang Peng +1
Decision-focused learning (DFL) has emerged as a powerful end-to-end alternative to conventional predict-then-optimize (PTO) pipelines by directly optimizing predictive models thro…
Estimating Real Demand Using a Flipped Queueing Model: A Case of Shared Micro-Mobility Services
Binyu Yang, Jinxiao Du, Junlin He +2
The spatial-temporal imbalance between supply and demand in shared micro-mobility services often leads to observed demand being censored, resulting in incomplete records of the und…
Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
Junlin He, Jinxiao Du, Wei Ma
Self-supervised learning (SSL) has rapidly advanced in recent years, approaching the performance of its supervised counterparts through the extraction of representations from unlab…
Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization
Junlin He, Jinxiao Du, Susu Xu +1
Multi-View Representation Learning (MVRL) aims to learn a unified representation of an object from multi-view data. Deep Canonical Correlation Analysis (DCCA) and its variants shar…