1 citations · 3 across the 23 of their papers we have counts for
25 papers · 1 filter
Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models
Ayan Sengupta, Vaibhav Seth, Tanmoy Chakraborty
Knowledge Distillation (KD) trains a smaller-capacity student model to imitate a larger-capacity teacher model by matching output distributions, implicitly assuming the teacher to…
Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization
Palaash Goel, Ayan Sengupta, Akshay Nambi +1
Structured pruning is a promising approach for compressing large language models (LLMs), yet existing methods rely heavily on greedy heuristics that produce myopic decisions, and o…
It Takes a MAESTRO To Prune Bad Experts
Palaash Goel, Ayush Maheshwari, Tanmoy Chakraborty
Sparsely-activated Mixture-of-Experts (MoE) language models achieve remarkable inference efficiency by activating only a small fraction of parameters per token, yet their full expe…
From Images to Words: Efficient Cross-Modal Knowledge Distillation to Language Models from Black-box Teachers
Ayan Sengupta, Shantanu Dixit, Md Shad Akhtar +1
Knowledge distillation (KD) methods are pivotal in compressing large pre-trained language models into smaller models, ensuring computational efficiency without significantly droppi…
Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Sampling
Suvadeep Hajra, Palash Nandi, Tanmoy Chakraborty
Safety tuning through supervised fine-tuning and reinforcement learning from human feedback has substantially improved the robustness of large language models. However, it typicall…
The Art of Scaling Test-Time Compute for Large Language Models
Aradhye Agarwal, Ayan Sengupta, Tanmoy Chakraborty
Test-time scaling (TTS) -- the dynamic allocation of compute during inference -- is a promising direction for improving reasoning in large language models (LLMs). However, a system…
Value-Guided KV Compression for LLMs via Approximated CUR Decomposition
Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty
Key-value (KV) cache compression has emerged as a critical technique for reducing the memory and latency overhead of autoregressive language models during inference. Prior approach…
Position: Enough of Scaling LLMs! Lets Focus on Downscaling
Yash Goel, Ayan Sengupta, Tanmoy Chakraborty
We challenge the dominant focus on neural scaling laws and advocate for a paradigm shift toward downscaling in the development of large language models (LLMs). While scaling laws h…
First Finish Search: Efficient Test-Time Scaling in Large Language Models
Aradhye Agarwal, Ayan Sengupta, Tanmoy Chakraborty
Test-time scaling (TTS), which involves dynamic allocation of compute during inference, offers a promising way to improve reasoning in large language models. While existing TTS met…
On the Generalization vs Fidelity Paradox in Knowledge Distillation
Suhas Kamasetty Ramesh, Ayan Sengupta, Tanmoy Chakraborty
Knowledge distillation (KD) is a key technique for compressing large language models into smaller ones while preserving performance. Despite the recent traction of KD research, its…
You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning
Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty
The ever-increasing size of large language models (LLMs) presents significant challenges for deployment due to their heavy computational and memory requirements. Current model prun…
How to Upscale Neural Networks with Scaling Law? A Survey and Practical Guidelines
Ayan Sengupta, Yash Goel, Tanmoy Chakraborty
Neural scaling laws have revolutionized the design and optimization of large-scale AI models by revealing predictable relationships between model size, dataset volume, and computat…
Sentiment-guided Commonsense-aware Response Generation for Mental Health Counseling
Aseem Srivastava, Gauri Naik, Alison Cerezo +2
The crisis of mental health issues is escalating. Effective counseling serves as a critical lifeline for individuals suffering from conditions like PTSD, stress, etc. Therapists fo…
Trust Modeling in Counseling Conversations: A Benchmark Study
Aseem Srivastava, Zuhair Hasan Shaik, Tanmoy Chakraborty +1
In mental health counseling, a variety of earlier studies have focused on dialogue modeling. However, most of these studies give limited to no emphasis on the quality of interactio…
Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarization
Aseem Srivastava, Smriti Joshi, Tanmoy Chakraborty +1
In mental health counseling, condensing dialogues into concise and relevant summaries (aka counseling notes) holds pivotal significance. Large Language Models (LLMs) exhibit remark…
Crowd Intelligence for Early Misinformation Prediction on Social Media
Megha Sundriyal, Harshit Choudhary, Tanmoy Chakraborty +1
Misinformation spreads rapidly on social media, causing serious damage by influencing public opinion, promoting dangerous behavior, or eroding trust in reliable sources. It spreads…
Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models
Aradhye Agarwal, Suhas K Ramesh, Ayan Sengupta +1
Fine-tuning large language models (LLMs) on downstream tasks requires substantial computational resources. Selective PEFT, a class of parameter-efficient fine-tuning (PEFT) methodo…
Intent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF
Amey Hengle, Aswini Kumar, Sahajpreet Singh +3
Counterspeech, defined as a response to mitigate online hate speech, is increasingly used as a non-censorial solution. Addressing hate speech effectively involves dispelling the st…
Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Study
Prottay Kumar Adhikary, Aseem Srivastava, Shivani Kumar +7
Comprehensive summaries of sessions enable an effective continuity in mental health counseling, facilitating informed therapy planning. Yet, manual summarization presents a signifi…
SemEval 2024 -- Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF)
Shivani Kumar, Md Shad Akhtar, Erik Cambria +1
We present SemEval-2024 Task 10, a shared task centred on identifying emotions and finding the rationale behind their flips within monolingual English and Hindi-English code-mixed…
Harmonizing Code-mixed Conversations: Personality-assisted Code-mixed Response Generation in Dialogues
Shivani Kumar, Tanmoy Chakraborty
Code-mixing, the blending of multiple languages within a single conversation, introduces a distinctive challenge, particularly in the context of response generation. Capturing the…
Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection
Sarah Masud, Mohammad Aflah Khan, Md. Shad Akhtar +1
As hate speech continues to proliferate on the web, it is becoming increasingly important to develop computational methods to mitigate it. Reactively, using black-box models to ide…
Overview of the CLAIMSCAN-2023: Uncovering Truth in Social Media through Claim Detection and Identification of Claim Spans
Megha Sundriyal, Md Shad Akhtar, Tanmoy Chakraborty
A significant increase in content creation and information exchange has been made possible by the quick development of online social media platforms, which has been very advantageo…
Manifold-Preserving Transformers are Effective for Short-Long Range Encoding
Ayan Sengupta, Md Shad Akhtar, Tanmoy Chakraborty
Multi-head self-attention-based Transformers have shown promise in different learning tasks. Albeit these models exhibit significant improvement in understanding short-term and lon…