71 citations · 182 across the 25 of their papers we have counts for
6 papers · 1 filter
Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
Dujian Ding, Ankur Mallick, Chi Wang +5
Large language models (LLMs) excel in most NLP tasks but also require expensive cloud servers for deployment due to their size, while smaller models that can be deployed on lower c…
Accelerating Dataset Distillation via Model Augmentation
Lei Zhang, Jie Zhang, Bowen Lei +6
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones. Existing DD methods based on gradient…
ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
Yue Zhao, Guoqing Zheng, Subhabrata Mukherjee +2
Existing works on anomaly detection (AD) rely on clean labels from human annotators that are expensive to acquire in practice. In this work, we propose a method to leverage weak/no…
Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners
Shashank Gupta, Subhabrata Mukherjee, Krishan Subudhi +4
Traditional multi-task learning (MTL) methods use dense networks that use the same set of shared weights across several different tasks. This often creates interference where two o…
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models
Mojan Javaheripi, Gustavo H. de Rosa, Subhabrata Mukherjee +6
The Transformer architecture is ubiquitously used as the building block of large-scale autoregressive language models. However, finding architectures with the optimal trade-off bet…
Fairness via Representation Neutralization
Mengnan Du, Subhabrata Mukherjee, Guanchu Wang +3
Existing bias mitigation methods for DNN models primarily work on learning debiased encoders. This process not only requires a lot of instance-level annotations for sensitive attri…