164 citations · 513 across the 30 of their papers we have counts for
33 papers
To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
Vignesh Kothapalli, Ata Fatahibaarzi, Hamed Firooz +1
Chain-of-thought (CoT) prompting combined with few-shot in-context learning (ICL) has unlocked significant reasoning capabilities in large language models (LLMs). However, ICL with…
Large Scale Retrieval for the LinkedIn Feed using Causal Language Models
Sudarshan Srinivasa Ramanujam, Antonio Alonso, Saurabh Kataria +20
In large scale recommendation systems like the LinkedIn Feed, the retrieval stage is critical for narrowing hundreds of millions of potential candidates to a manageable subset for…
CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations
Vignesh Kothapalli, Hamed Firooz, Maziar Sanjabi
We introduce CoT-ICL Lab, a framework and methodology to generate synthetic tokenized datasets and systematically study chain-of-thought (CoT) in-context learning (ICL) in language…
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song +17
Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Al…
360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation
Hamed Firooz, Maziar Sanjabi, Adrian Englhardt +20
Ranking and recommendation systems are the foundation for numerous online experiences, ranging from search results to personalized content delivery. These systems have evolved into…
Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks
Hamed Firooz, Maziar Sanjabi, Wenlong Jiang +1
Despite significant advancements, Large Language Models (LLMs) exhibit blind spots that impair their ability to retrieve and process relevant contextual data effectively. We demons…
Enhancing Stability for Large Language Models Training in Constrained Bandwidth Networks
Yun Dai, Tejas Dharamsi, Byron Hsu +2
Training extremely large language models (LLMs) with billions of parameters is a computationally intensive task that pushes the limits of current data parallel training systems. Wh…
Resprompt: Residual Connection Prompting Advances Multi-Step Reasoning in Large Language Models
Song Jiang, Zahra Shakeri, Aaron Chan +8
Chain-of-thought (CoT) prompting, which offers step-by-step problem-solving rationales, has impressively unlocked the reasoning potential of large language models (LLMs). Yet, the…
Identifying Interpretable Subspaces in Image Representations
Neha Kalibhat, Shweta Bhardwaj, Bayan Bruss +3
We propose Automatic Feature Explanation using Contrasting Concepts (FALCON), an interpretability framework to explain features of image representations. For a target feature, FALC…
Meta-training with Demonstration Retrieval for Efficient Few-shot Learning
Aaron Mueller, Kanika Narang, Lambert Mathias +2
Large language models show impressive results on few-shot NLP tasks. However, these models are memory and computation-intensive. Meta-training allows one to leverage smaller models…
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning
Ajinkya Tejankar, Maziar Sanjabi, Qifan Wang +4
Recently, self-supervised learning (SSL) was shown to be vulnerable to patch-based data poisoning backdoor attacks. It was shown that an adversary can poison a small part of the un…
COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation
Nan Wang, Qifan Wang, Yi-Chia Wang +5
As language models become increasingly integrated into our digital lives, Personalized Text Generation (PTG) has emerged as a pivotal component with a wide range of applications. H…
FRAME: Evaluating Rationale-Label Consistency Metrics for Free-Text Rationales
Aaron Chan, Shaoliang Nie, Liang Tan +4
Following how humans communicate, free-text rationales aim to use natural language to explain neural language model (LM) behavior. However, free-text rationales' unconstrained natu…
Detecting and Understanding Harmful Memes: A Survey
Shivam Sharma, Firoj Alam, Md. Shad Akhtar +7
The automatic identification of harmful content online is of major concern for social media platforms, policymakers, and society. Researchers have studied textual, visual, and audi…
ER-Test: Evaluating Explanation Regularization Methods for Language Models
Brihi Joshi, Aaron Chan, Ziyi Liu +4
By explaining how humans would solve a given task, human rationales can provide strong learning signal for neural language models (LMs). Explanation regularization (ER) aims to imp…
Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem
Khalil Mrini, Shaoliang Nie, Jiatao Gu +3
We propose an autoregressive entity linking model, that is trained with two auxiliary tasks, and learns to re-rank generated samples at inference time. Our proposed novelties addre…
Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features
Neha Kalibhat, Kanika Narang, Hamed Firooz +2
Self-supervised learning (SSL) has shown impressive results in downstream classification tasks. However, there is limited work in understanding their failure modes and interpreting…
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision
Ajinkya Tejankar, Maziar Sanjabi, Bichen Wu +4
Using natural language as a supervision for training visual recognition models holds great promise. Recent works have shown that if such supervision is used in the form of alignmen…
BARACK: Partially Supervised Group Robustness With Guarantees
Nimit S. Sohoni, Maziar Sanjabi, Nicolas Ballas +4
While neural networks have shown remarkable success on classification tasks in terms of average-case performance, they often fail to perform well on certain groups of the data. Suc…
UNIREX: A Unified Learning Framework for Language Model Rationale Extraction
Aaron Chan, Maziar Sanjabi, Lambert Mathias +5
An extractive rationale explains a language model's (LM's) prediction on a given task instance by highlighting the text inputs that most influenced the prediction. Ideally, rationa…
Detecting Propaganda Techniques in Memes
Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar +5
Propaganda can be defined as a form of communication that aims to influence the opinions or the actions of people towards a specific goal; this is achieved by means of well-defined…
SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images
Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar +5
We describe SemEval-2021 task 6 on Detection of Persuasion Techniques in Texts and Images: the data, the annotation guidelines, the evaluation setup, the results, and the participa…
A Survey on Multimodal Disinformation Detection
Firoj Alam, Stefano Cresci, Tanmoy Chakraborty +6
Recent years have witnessed the proliferation of offensive content online such as fake news, propaganda, misinformation, and disinformation. While initially this was mostly about t…
MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding
Woojeong Jin, Maziar Sanjabi, Shaoliang Nie +3
To reduce a model size but retain performance, we often rely on knowledge distillation (KD) which transfers knowledge from a large "teacher" model to a smaller "student" model. How…