208 citations · 369 across the 38 of their papers we have counts for
19 papers · 1 filter
Activation-Conditioned Self-Distillation
Zhexi Lu, Subhajit Chaudhury, Tejaswini Pedapati +2
On-policy self-distillation uses a model as its own teacher to provide dense supervision for reasoning, often through reference-solution conditioning. Providing privileged informat…
ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
David H. Yang, Yuxuan Zhu, Mohammad Mohammadi Amiri +4
Large language models (LLMs) have shown great performance on complex reasoning tasks but often require generating long intermediate thoughts before reaching a final answer. During…
CoFrNets: Interpretable Neural Architecture Inspired by Continued Fractions
Isha Puri, Amit Dhurandhar, Tejaswini Pedapati +3
In recent years there has been a considerable amount of research on local post hoc explanations for neural networks. However, work on building interpretable neural architectures ha…
Sparse Gradient Compression for Fine-Tuning Large Language Models
David H. Yang, Mohammad Mohammadi Amiri, Tejaswini Pedapati +2
Fine-tuning large language models (LLMs) for downstream tasks has become increasingly crucial due to their widespread use and the growing availability of open-source models. Howeve…
Differentiable Prompt Learning for Vision Language Models
Zhenhan Huang, Tejaswini Pedapati, Pin-Yu Chen +1
Prompt learning is an effective way to exploit the potential of large-scale pre-trained foundational models. Continuous prompts parameterize context tokens in prompts by turning th…
Large Language Models can be Strong Self-Detoxifiers
Ching-Yun Ko, Pin-Yu Chen, Payel Das +6
Reducing the likelihood of generating harmful and toxic output is an essential task when aligning large language models (LLMs). Existing methods mainly rely on training an external…
TabSketchFM: Sketch-based Tabular Representation Learning for Data Discovery over Data Lakes
Aamod Khatiwada, Harsha Kokel, Ibrahim Abdelaziz +7
Enterprises have a growing need to identify relevant tables in data lakes; e.g. tables that are unionable, joinable, or subsets of each other. Tabular neural models can be helpful…
Graph is all you need? Lightweight data-agnostic neural architecture search without training
Zhenhan Huang, Tejaswini Pedapati, Pin-Yu Chen +2
Neural architecture search (NAS) enables the automatic design of neural network models. However, training the candidates generated by the search algorithm for performance evaluatio…
NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models
Amit Dhurandhar, Tejaswini Pedapati, Ronny Luss +4
Transformer-based Language Models have become ubiquitous in Natural Language Processing (NLP) due to their impressive performance on various tasks. However, expensive training as w…
From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers
Bharat Runwal, Tejaswini Pedapati, Pin-Yu Chen
Pretrained Language Models (PLMs) have become the de facto starting point for fine-tuning on downstream tasks. However, as model sizes continue to increase, traditional fine-tuning…
MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning
Krishnateja Killamsetty, Alexandre V. Evfimievski, Tejaswini Pedapati +3
Training deep networks and tuning hyperparameters on large datasets is computationally intensive. One of the primary research directions for efficient training is to reduce trainin…
Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics
Chunheng Jiang, Tejaswini Pedapati, Pin-Yu Chen +2
Efficient model selection for identifying a suitable pre-trained neural network to a downstream task is a fundamental yet challenging task in deep learning. Current practice requir…
Multihop: Leveraging Complex Models to Learn Accurate Simple Models
Amit Dhurandhar, Tejaswini Pedapati
Knowledge transfer from a complex high performing model to a simpler and potentially low performing one in order to enhance its performance has been of great interest over the last…
Learning to Rank Learning Curves
Martin Wistuba, Tejaswini Pedapati
Many automated machine learning methods, such as those for hyperparameter and neural architecture optimization, are computationally expensive because they involve training many dif…
Learning Global Transparent Models Consistent with Local Contrastive Explanations
Tejaswini Pedapati, Avinash Balakrishnan, Karthikeyan Shanmugam +1
There is a rich and growing literature on producing local contrastive/counterfactual explanations for black-box models (e.g. neural networks). In these methods, for an input, an ex…
A Survey on Neural Architecture Search
Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati
The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architectu…
Model Agnostic Contrastive Explanations for Structured Data
Amit Dhurandhar, Tejaswini Pedapati, Avinash Balakrishnan +3
Recently, a method [7] was proposed to generate contrastive explanations for differentiable models such as deep neural networks, where one has complete access to the model. In this…
Inductive Transfer for Neural Architecture Optimization
Martin Wistuba, Tejaswini Pedapati
The recent advent of automated neural network architecture search led to several methods that outperform state-of-the-art human-designed architectures. However, these approaches ar…
NeuNetS: An Automated Synthesis Engine for Neural Network Design
Atin Sood, Benjamin Elder, Benjamin Herta +17
Application of neural networks to a vast variety of practical applications is transforming the way AI is applied in practice. Pre-trained neural network models available through AP…