2.9k citations · 3.2k across the 15 of their papers we have counts for
19 papers
Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)
Krishnaram Kenthapadi, Mehrnoosh Sameki, Ankur Taly
With the ongoing rapid adoption of Artificial Intelligence (AI)-based systems in high-stakes domains, ensuring the trustworthiness, safety, and observability of these systems has b…
An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh +1
Incrementally fine-tuning foundational models on new tasks or domains is now the de facto approach in NLP. A known pitfall of this approach is the \emph{catastrophic forgetting} of…
Identifying and Mitigating the Security Risks of Generative AI
Clark Barrett, Brad Boyd, Elie Burzstein +20
Every major technical invention resurfaces the dual-use dilemma -- the new technology has the potential to be used for good as well as for harm. Generative AI (GenAI) techniques, s…
Interpretable Mixture of Experts
Aya Abdelsalam Ismail, Sercan Ö. Arik, Jinsung Yoon +3
The need for reliable model explanations is prominent for many machine learning applications, particularly for tabular and time-series data as their use cases often involve high-st…
First is Better Than Last for Language Data Influence
Chih-Kuan Yeh, Ankur Taly, Mukund Sundararajan +2
The ability to identify influential training examples enables us to debug training data and explain model behavior. Existing techniques to do so are based on the flow of training d…
Local Explanations via Necessity and Sufficiency: Unifying Theory and Practice
David Watson, Limor Gultchin, Ankur Taly +1
Necessity and sufficiency are the building blocks of all successful explanations. Yet despite their importance, these notions have been conceptually underdeveloped and inconsistent…
The Explanation Game: Explaining Machine Learning Models Using Shapley Values
Luke Merrick, Ankur Taly
A number of techniques have been proposed to explain a machine learning model's prediction by attributing it to the corresponding input features. Popular among these are techniques…
Explainable Machine Learning in Deployment
Umang Bhatt, Alice Xiang, Shubham Sharma +7
Explainable machine learning offers the potential to provide stakeholders with insights into model behavior by using various methods such as feature importance scores, counterfactu…
Property Inference for Deep Neural Networks
Divya Gopinath, Hayes Converse, Corina S. Pasareanu +1
We present techniques for automatically inferring formal properties of feed-forward neural networks. We observe that a significant part (if not all) of the logic of feed forward ne…
Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry
Kevin McCloskey, Ankur Taly, Federico Monti +2
Deep neural networks have achieved state of the art accuracy at classifying molecules with respect to whether they bind to specific protein targets. A key breakthrough would occur…
Counterfactual Fairness in Text Classification through Robustness
Sahaj Garg, Vincent Perot, Nicole Limtiaco +3
In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the exampl…
A Note about: Local Explanation Methods for Deep Neural Networks lack Sensitivity to Parameter Values
Mukund Sundararajan, Ankur Taly
Local explanation methods, also known as attribution methods, attribute a deep network's prediction to its input (cf. Baehrens et al. (2010)). We respond to the claim from Adebayo…
Did the Model Understand the Question?
Pramod Kaushik Mudrakarta, Ankur Taly, Mukund Sundararajan +1
We analyze state-of-the-art deep learning models for three tasks: question answering on (1) images, (2) tables, and (3) passages of text. Using the notion of \emph{attribution} (wo…
It was the training data pruning too!
Pramod Kaushik Mudrakarta, Ankur Taly, Mukund Sundararajan +1
We study the current best model (KDG) for question answering on tabular data evaluated over the WikiTableQuestions dataset. Previous ablation studies performed against this model a…
Abductive Matching in Question Answering
Kedar Dhamdhere, Kevin S. McCurley, Mukund Sundararajan +1
We study question-answering over semi-structured data. We introduce a new way to apply the technique of semantic parsing by applying machine learning only to provide annotations th…
Axiomatic Attribution for Deep Networks
Mukund Sundararajan, Ankur Taly, Qiqi Yan
We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms-…
Gradients of Counterfactuals
Mukund Sundararajan, Ankur Taly, Qiqi Yan
Gradients have been used to quantify feature importance in machine learning models. Unfortunately, in nonlinear deep networks, not only individual neurons but also the whole networ…
Distributed Authorization in Vanadium
Andres Erbsen, Asim Shankar, Ankur Taly
In this tutorial, we present an authorization model for distributed systems that operate with limited internet connectivity. Reliable internet access remains a luxury for a majorit…
Privacy, Discovery, and Authentication for the Internet of Things
David J. Wu, Ankur Taly, Asim Shankar +1
Automatic service discovery is essential to realizing the full potential of the Internet of Things (IoT). While discovery protocols like Multicast DNS, Apple AirDrop, and Bluetooth…