9 citations · 26 across the 14 of their papers we have counts for
15 papers
A Statistical Perspective on Knowledge Distillation: Foundations, Classical Methods, and Large Language Model Extensions
Luyang Fang, Haoran Lu, Jiazhang Cai +4
Knowledge Distillation (KD) has emerged as a vital paradigm for transferring the capabilities of high-capacity models to efficient ``student'' counterparts, addressing critical cha…
Complex Problem Solving in Large Language Models: A Statistical Control Survey and Diagnostic Framework
Jiazhang Cai, Tao Wang, Ruidong Zhang +35
Complex problem solving (CPS) with large language models (LLMs) is often framed as a matter of stronger reasoning or longer generation. Yet early-step error amplification, prompt b…
Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation
Luyang Fang, Haoran Lu, Yongkai Chen +2
As machine learning models and datasets continue to grow, developing complex models has become increasingly computationally demanding. Knowledge distillation reduces deployment cos…
Confidence-Aware Automated Assessment of Student-Drawn Scientific Models
Luyang Fang, Yingchuan Zhang, Jongchan Park +3
Student-generated drawings are widely used in science education to assess learners' conceptual understanding in modeling-based tasks aligned with the Next Generation Science Standa…
Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
Luyang Fang, Yongkai Chen, Jiazhang Cai +2
Knowledge distillation is a powerful method for model compression, enabling the efficient deployment of complex deep learning models (teachers), including large language models. Ho…
NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning
Haoran Lu, Luyang Fang, Wenxuan Zhong +1
Multi-agent language systems are often built as hand-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance. We propose N…
Generalizable and Efficient Automated Scoring with a Knowledge-Distilled Multi-Task Mixture-of-Experts
Luyang Fang, Tao Wang, Ping Ma +1
Automated scoring of written constructed responses typically relies on separate models per task, straining computational resources, storage, and maintenance in real-world education…
DCMM-Transformer: Degree-Corrected Mixed-Membership Attention for Medical Imaging
Huimin Cheng, Xiaowei Yu, Shushan Wu +7
Medical images exhibit latent anatomical groupings, such as organs, tissues, and pathological regions, that standard Vision Transformers (ViTs) fail to exploit. While recent work l…
Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
Haoran Lu, Luyang Fang, Ruidong Zhang +47
Due to the remarkable capabilities and growing impact of large language models (LLMs), they have been deeply integrated into many aspects of society. Thus, ensuring their alignment…
Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
Luyang Fang, Xiaowei Yu, Jiazhang Cai +23
The exponential growth of Large Language Models (LLMs) continues to highlight the need for efficient strategies to meet ever-expanding computational and data demands. This survey p…
Efficient Multi-Task Inferencing: Model Merging with Gromov-Wasserstein Feature Alignment
Luyang Fang, Ehsan Latif, Haoran Lu +3
Automatic scoring of student responses enhances efficiency in education, but deploying a separate neural network for each task increases storage demands, maintenance efforts, and r…
Large Language Models for Bioinformatics
Wei Ruan, Yanjun Lyu, Jing Zhang +52
With the rapid advancements in large language model (LLM) technology and the emergence of bioinformatics-specific language models (BioLMs), there is a growing need for a comprehens…
A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Education
Ehsan Latif, Yifan Zhou, Shuchen Guo +24
As artificial intelligence (AI) continues to advance, it demonstrates capabilities comparable to human intelligence, with significant potential to transform education and workforce…
Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
Ehsan Latif, Luyang Fang, Ping Ma +1
This study proposes a method for knowledge distillation (KD) of fine-tuned Large Language Models (LLMs) into smaller, more efficient, and accurate neural networks. We specifically…
Using GPT-4 to Augment Unbalanced Data for Automatic Scoring
Luyang Fang, Gyeong-Geon Lee, Xiaoming Zhai
Machine learning-based automatic scoring faces challenges with unbalanced student responses across scoring categories. To address this, we introduce a novel text data augmentation…