# knowledge distillation

#multimodal reasoning#chain-of-thought#latent representation#visual reasoning

OPLD: On-Policy Latent Distillation for Multimodal Reasoning

Shoutai Zhu, Tianyang Xu, Bin Sun +3

The paper introduces OPLD, an on‑policy latent distillation framework that transfers the reasoning ability of privileged multimodal chain‑of‑thought prompts into continuous latent…

cs.AI2026

Correcting What You Cannot See: Credit Assignment for Perception Distillation in Multimodal Reasoners

Feng Xiong, Leyan Xue, Hongyu Lin

The paper proposes Perception-Correction Distillation (PCD), a label‑free method that uses downstream failures and teacher‑student disagreement to pinpoint and correct perception e…

#multimodal reasoning#knowledge distillation#perception correction#teacher-student disagreement

cs.IR2026

From Understanding to Action: Feedback-Grounded Policy Discovery for Generative Recommendation

Zhi Chen, Minmao Wang, Xingchen Liu +8

The paper introduces a feedback‑driven framework that first extracts user intent and then discovers recommendation policies using outcome‑derived feedback, distilling this knowledg…

#generative recommendation#large language models#feedback‑driven policy discovery#intent modeling

#adversarial robustness#knowledge distillation#information bottleneck#dual teacher

Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers

Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi +1

The paper extends information bottleneck distillation by adding a clean‑trained teacher alongside a robust teacher, using cross‑layer attention to improve both clean accuracy and a…

#semi-supervised learning#molecular graphs#graph neural networks#ensemble methods

Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus

Rasmus Tirsgaard, Laurits Fredsgaard, Marisa Wodrich +2

The paper proposes a semi-supervised learning approach for molecular graph data that uses an ensemble consensus objective to improve prediction accuracy, robustness, and calibratio…

#knowledge distillation#heterogeneous models#redundancy suppression#feature decorrelation

CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation

Fengming Yu, Haiwei Pan, Kejia Zhang +3

The paper introduces CoCaRS, a method for heterogeneous knowledge distillation that calibrates feature decorrelation to suppress redundancy while preserving structural information,…

#fake news detection#multimodal learning#domain adaptation#knowledge distillation

Where Detectors Fail: Closing the Tail-Domain Gap with Expert-Guided Mutual Distillation

Xuan Feng, Guihong Liu, Tianlong Gu +5

The paper introduces Expert-Guided Mutual Distillation (EGMD), a method that improves multimodal fake news detection across domains by calibrating input coherence, aligning domain…

#knowledge distillation#diabetic retinopathy screening#edge deployment#heterogeneous teachers

OrthKD: Extracting Generalized Clinical Knowledge from Heterogeneous Teachers for Lightweight Deployment

Yi Xu, Cheng Chen, Mufan Cao

The paper introduces OrthKD, a knowledge distillation framework that selectively trusts a strong CNN and a weaker transformer teacher to train a lightweight MobileNetV3 model for d…

cs.AI2026

Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis

Wanxu Cai, Zhengyu Chen, Huaisheng Zhu +3

The paper introduces a time‑truncation harness that limits temporal information during data synthesis, enabling large language models to perform more effective temporal search and…

#future event prediction#temporal reasoning#large language models#data synthesis

#image classification#edge computing#knowledge distillation#dataset expansion

Lightweight Image Classification of Raptor Species for Edge Devices: Rare-Species Dataset Expansion via Video Frame Extraction, Knowledge Distillation, and TensorRT Deployment

Takeshi Nishikawa

The paper presents a lightweight model ensemble for classifying raptor species on edge devices, using knowledge distillation from a large teacher model and expanding the dataset vi…

#on-policy distillation#large language models#knowledge distillation#reinforcement learning

Weak-to-Strong On-Policy Distillation

Fangxu Yu, Zinan Lin, Xiaodong Liu +4

The paper proposes Weak-to-Strong On-Policy Distillation (W2S-OPD), a method that improves a large language model by distilling knowledge from multiple weaker models using a constr…

#chain of thought#knowledge distillation#large language models#answer conditioning

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

Jungseob Lee, Seungyoon Lee, Suhyune Son +4

The paper shows that conditioning large language models on the correct answer when generating chains of thought harms the quality of distilled reasoning data, leading to large drop…

#multiple instance learning#knowledge distillation#computational pathology#pretraining

Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models

Mingxi Fu, Jiawen Li, Renao Yan +4

The paper introduces a distillation-based pretraining framework that transfers knowledge from two slide-level foundation models into multiple instance learning (MIL) networks for w…

#video object removal#knowledge distillation#one-step generation#temporal transformer

From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting

Zizhao Chen, Ping Wei, Guang Dai +2

The paper introduces D2DF, a one‑step video object removal framework that learns to turn coarse removal drafts into high‑quality videos via privileged distillation, and adds a self…

#tool use#on-policy distillation#knowledge distillation#model calibration

When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

Jiabin Shen, Guang Chen, Chengjun Mao

The paper studies how multi-teacher on-policy distillation can cause language models to over-call tools, and introduces Soft Clamp, a token-level divergence calibration method that…

#incremental learning#object detection#knowledge distillation#symbiosis

Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

Mingyue Zeng, De Cheng, Zhipeng Xu +3

The paper introduces Symbiosis-Inspired Knowledge Distillation (SIKD), a method for incremental object detection that leverages spatial and semantic relationships between old and n…

#handwritten character recognition#devanagari script#parameter-efficient models#knowledge distillation

Barnamala: Parameter-Efficient Handwritten Devanagari Recognition at Benchmark Saturation

Ashish Thapa, Samrat Karki

The paper presents a compact 1.11 M‑parameter convolutional network for handwritten Devanagari character recognition that achieves 99.73% accuracy, matching larger models while bei…

cs.IR2026

LLM-Based User Personas for Recommendations at Scale

Haoting Wang, Haokai Lu, Zheyun Feng +14

The paper presents a framework that uses large language models to generate natural-language user interest personas in real time for a large‑scale video recommendation system, emplo…

#large language models#user personas#real-time recommendation#knowledge distillation

#large language models#model alignment#plug-and-play adaptation#knowledge distillation

Decoupled Alignment for Robust Plug-and-Play Adaptation

Haozheng Luo, Jiahao Yu, Wenxin Zhang +9

The paper proposes a training-free, plug-and-play method that uses knowledge distillation and model fusion to correct misaligned (shadow-aligned) large language models, improving s…

#segment anything#mobile vision#knowledge distillation#hypergraph

MobileSAM2: Lightweight Segment Anything for Spatial Intelligence

Kai Jiang, Jiaxing Huang, Jingyi Zhang +5

The paper introduces MobileSAM2, a lightweight version of the SAM2 segmentation model designed for mobile devices, using hypergraph-based knowledge distillation to transfer tempora…

#gastric endoscopy#neoplasm classification#color-invariant learning#knowledge distillation

MAGE: Color-Invariant and Spatial Knowledge Distillation for Gastric Neoplasm Classification

Jiho Jun, Jeongwon Woo, Jaemin Song +6

The paper introduces MAGE, a framework that uses masked achromatic views and dual-objective knowledge distillation to train a model that classifies gastric adenoma versus carcinoma…

cs.AI2026

Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?

Kaiwen Zheng, Junchen Fu, Wenhao Deng +3

The paper introduces Light-MER, a sub‑billion‑parameter multimodal emotion recognition model that uses knowledge distillation, an optimal transport loss, and a multi‑reward optimiz…

#multimodal emotion recognition#knowledge distillation#lightweight models#optimal transport loss

#domain incremental learning#remote sensing change detection#knowledge distillation#frequency domain processing

Domain-Incremental Remote Sensing Change Detection via Difference-Guided Adaptation and Frequency-Decoupled Distillation

Daifeng Peng, Yaning Li, Haiyan Guan

The paper introduces DG-FDD, a framework for domain‑incremental remote sensing change detection that uses a difference‑guided adapter to capture bitemporal discrepancies and a freq…