2 citations · 5 across the 4 of their papers we have counts for
4 papers · 1 filter
INDUS: Effective and Efficient Language Models for Scientific Applications
Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka +33
Large language models (LLMs) trained on general domain corpora showed remarkable results on natural language processing (NLP) tasks. However, previous research demonstrated LLMs tr…
Neural Architecture Search for Effective Teacher-Student Knowledge Transfer in Language Models
Aashka Trivedi, Takuma Udagawa, Michele Merler +3
Large pretrained language models have achieved state-of-the-art results on a variety of downstream tasks. Knowledge Distillation (KD) into a smaller student model addresses their i…
Zero-Shot Dynamic Quantization for Transformer Inference
Yousef El-Kurdi, Jerry Quinn, Avirup Sil
We introduce a novel run-time method for significantly reducing the accuracy loss associated with quantizing BERT-like models to 8-bit integers. Existing methods for quantizing mod…
Multi-Granular Text Encoding for Self-Explaining Categorization
Zhiguo Wang, Yue Zhang, Mo Yu +5
Self-explaining text categorization requires a classifier to make a prediction along with supporting evidence. A popular type of evidence is sub-sequences extracted from the input…