8 papers
T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning
Yanjun Fu, Faisal Hamman, Sanghamitra Dutta
Instruction tuning is essential for Large Language Models (LLMs) to effectively follow user instructions. To improve training efficiency and reduce data redundancy, recent works us…
Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition
Barproda Halder, Faisal Hamman, Pasan Dissanayake +3
Spuriousness arises when there is an association between two or more variables in a dataset that are not causally related. In this work, we propose an explainability framework to p…
Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
Faisal Hamman, Pasan Dissanayake, Yanjun Fu +1
Knowledge distillation is a promising approach to transfer capabilities from complex teacher models to smaller, resource-efficient student models that can be deployed easily, parti…
Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards
Faisal Hamman, Chenyang Zhu, Anoop Kumar +4
RAG systems are increasingly deployed in high-stakes domains where users expect outputs to be consistent across semantically equivalent queries. However, existing systems often exh…
Quantifying Prediction Consistency Under Fine-Tuning Multiplicity in Tabular LLMs
Faisal Hamman, Pasan Dissanayake, Saumitra Mishra +2
Fine-tuning LLMs on tabular classification tasks can lead to the phenomenon of fine-tuning multiplicity where equally well-performing models make conflicting predictions on the sam…
Quantifying Knowledge Distillation Using Partial Information Decomposition
Pasan Dissanayake, Faisal Hamman, Barproda Halder +3
Knowledge distillation deploys complex machine learning models in resource-constrained environments by training a smaller student model to emulate internal representations of a com…