5 papers
How Far Can Sub-3B Open Language Models Go in Zero-Shot Essay Scoring on an 8 GB Consumer GPU?
Nguyen Dung Son, Dang Quang Minh, Nguyen Huu Loi +2
Zero-shot essay scoring with large language models is usually demonstrated with proprietary API models, yet the settings where automated scoring is most needed, such as public scho…
Accuracy Is Not Enough: A Cross-Architecture Audit of Demographic Bias in Deep Knowledge Tracing
Dang Quang Minh, Nguyen Dung Son, Nguyen Huu Loi +2
Deep knowledge tracing (DKT) models implicitly decide which students an adaptive system believes have mastered a skill, yet almost all evidence on their demographic fairness comes…
BRIDGE: Budget-aware Reasoning via Intermediate Distillation with Guided Examples
Xuan-An Le, Minh-Nam Tran, Son Nguyen
Distilling knowledge from large proprietary models (e.g., GPT-4) to tiny deployable models (less than 1B parameters) faces a critical capacity-budget trap: the 1000x capacity gap b…
Improving Adaptive Moment Optimization via Preconditioner Diagonalization
Son Nguyen, Bo Liu, Lizhang Chen +1
Modern adaptive optimization methods, such as Adam and its variants, have emerged as the most widely used tools in deep learning over recent years. These algorithms offer automatic…
Memory-Efficient Optimization with Factorized Hamiltonian Descent
Son Nguyen, Lizhang Chen, Bo Liu +1
Modern deep learning heavily depends on adaptive optimizers such as Adam and its variants, which are renowned for their capacity to handle model scaling and streamline hyperparamet…