13 papers
OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning
Mohamed Bouadi, Nassim Bouarour, Aditya Tanna +1
Tabular foundation models enable in-context learning by predicting unseen observations from a labeled support table without task-specific parameter updates. Despite recent advances…
Efficient LLM Distillation for Bangladesh Legal Context: A Smartphone-Compatible Retrieval-Augmented Generation Model
MD. Nafis Kamal, Mahadi Hasan Fahim, Talha Ridwan +4
Legal information in Bangladesh is inaccessible to most citizens. Statutory text is English-only, trained lawyers are concentrated in urban centres, and cloud-dependent AI fails wh…
Below the Noise Floor: Bimodal Seed Collapse and Distinct Failure Modes in Small-Model Knowledge Distillation
Dipto Sumit, Sakib Ul Haque, Farig Sadeque
Function routing -- selecting the correct API call from a fixed catalog given a natural-language request -- is a deployment problem where small students are attractive but knowledg…
BengaliMCQ: Automatic Generation and Answer Prediction of Academic Multiple-Choice Questions in a Low-Resource Language
Abu Tarabin Surzo, A. K. M. Nihalul Kabir, Sm Azmain Faysal +3
Traditional retrieval-augmented generation (RAG) frameworks process documents without attending to their hierarchical structure, leading to poor performance, especially in low-reso…
Characterizing Human-Likeness in AI Generated Poetry: A Zero-shot Classification Study
A. N. Biswas, T. Tabassum, A. A. Shohid +4
With the advancement of AI technologies, Generative AI (GenAI) and human written text have become nearly indistinguishable. Additionally, the global standardization of AI chatbots…
When Does Knowledge Distillation Hurt? Reliability-Aware Distillation for Low-Resource Language Summarization
Dipto Sumit, Ankan Kumar Roy Srizon, Sadia Khair Rodela +4
Knowledge distillation (KD) is a standard approach for compressing sequence-to-sequence models, but its per-sample effects are rarely examined. On the BanSum Bangla summarization b…
Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing
Ariyan Hossain, Kazi Kamruzzaman Rabbi, Farig Sadeque +1
Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by…
Machine Unlearning for Class Removal through SISA-based Deep Neural Network Architectures
Ishrak Hamim Mahi, Siam Ferdous, Md Sakib Sadman Badhon +4
The rapid proliferation of image generation models and other artificial intelligence (AI) systems has intensified concerns regarding data privacy and user consent. As the availabil…
Exploring the Limits of Pruning: Task-Specific Neurons, Model Collapse, and Recovery in Task-Specific Large Language Models
M. K. Khalidi Siam, Md. Tausif-Ul-Islam, Md. Reshad Romim Khan +5
Neuron pruning is widely used to reduce the computational cost and parameter footprint of large language models, yet it remains unclear whether neurons in task-specific models cont…
Enhancing Mental Health Counseling Support in Bangladesh using Culturally-Grounded Knowledge
Md Arid Hasan, Azhagu Meena SP, Aditya Khan +6
Large language models (LLMs) show promise in generating supportive responses for mental health and counseling applications. However, their responses often lack cultural sensitivity…
Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarization
Dipto Sumit, Ankan Kumar Roy, Sadia Khair Rodela +4
We study multiteacher knowledge distillation for low resource abstractive summarization from a reliability aware perspective. We introduce EWAD (Entropy Weighted Agreement Aware Di…
Benchmarking Bengali Dialectal Bias: A Multi-Stage Framework Integrating RAG-Based Translation and Human-Augmented RLAIF
K. M. Jubair Sami, Dipto Sumit, Ariyan Hossain +1
Large language models (LLMs) frequently exhibit performance biases against regional dialects of low-resource languages. However, frameworks to quantify these disparities remain sca…
Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure
Syed Naveed Mahmood, Md. Rezaur Rahman Bhuiyan, Tasfia Zaman +5
Entity-level unlearning is usually evaluated by what a model says: whether it stops naming the target, refuses a query, or shifts a Truth Ratio distribution. These output-level tes…