100 citations · 245 across the 24 of their papers we have counts for
25 papers
Recirculation
Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer +2
We describe an inference-time architectural enhancement for off-the-shelf foundation models that systematically reduces perplexity and boosts accuracy across generation and reasoni…
The Topological Trouble With Transformers
Michael C. Mozer, Shoaib Ahmed Siddiqui, Rosanne Liu
Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tr…
Position: Capability Control Should be a Separate Goal From Alignment
Shoaib Ahmed Siddiqui, Eleni Triantafillou, David Krueger +1
Foundation models are trained on broad data distributions, yielding generalist capabilities that enable many downstream applications but also expand the space of potential misuse a…
From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization
Shoaib Ahmed Siddiqui, Adrian Weller, David Krueger +3
Recent unlearning methods for LLMs are vulnerable to relearning attacks: knowledge believed-to-be-unlearned re-emerges by fine-tuning on a small set of (even seemingly-unrelated) e…
JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs
Aliaksandra Shysheya, John Bronskill, James Requeima +4
We introduce a simple method for probabilistic predictions on tabular data based on Large Language Models (LLMs) called JoLT (Joint LLM Process for Tabular data). JoLT uses the in-…
Exploring the design space of deep-learning-based weather forecasting systems
Shoaib Ahmed Siddiqui, Jean Kossaifi, Boris Bonev +4
Despite tremendous progress in developing deep-learning-based weather forecasting systems, their design space, including the impact of different design choices, is yet to be well u…
On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective
Shoaib Ahmed Siddiqui, Yanzhi Chen, Juyeon Heo +2
Recent works have successfully applied Large Language Models (LLMs) to function modeling tasks. However, the reasons behind this success remain unclear. In this work, we propose a…
Permissive Information-Flow Analysis for Large Language Models
Shoaib Ahmed Siddiqui, Radhika Gaonkar, Boris Köpf +7
Large Language Models (LLMs) are rapidly becoming commodity components of larger software systems. This poses natural security and privacy problems: poisoned data retrieved from on…
Protecting against simultaneous data poisoning attacks
Neel Alex, Shoaib Ahmed Siddiqui, Amartya Sanyal +1
Current backdoor defense methods are evaluated against a single attack at a time. This is unrealistic, as powerful machine learning systems are trained on large datasets scraped fr…
A deeper look at depth pruning of LLMs
Shoaib Ahmed Siddiqui, Xin Dong, Greg Heinrich +4
Large Language Models (LLMs) are not only resource-intensive to train but even more costly to deploy in production. Therefore, recent work has attempted to prune blocks of LLMs bas…
Investigating the Nature of 3D Generalization in Deep Neural Networks
Shoaib Ahmed Siddiqui, David Krueger, Thomas Breuel
Visual object recognition systems need to generalize from a set of 2D training views to novel views. The question of how the human visual system can generalize to novel views has b…
Blockwise Self-Supervised Learning at Scale
Shoaib Ahmed Siddiqui, David Krueger, Yann LeCun +1
Current state-of-the-art deep networks are all powered by backpropagation. In this paper, we explore alternatives to full backpropagation in the form of blockwise learning rules, l…
Domain Generalization for Robust Model-Based Offline Reinforcement Learning
Alan Clark, Shoaib Ahmed Siddiqui, Robert Kirk +3
Existing offline reinforcement learning (RL) algorithms typically assume that training data is either: 1) generated by a known policy, or 2) of entirely unknown origin. We consider…
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics
Shoaib Ahmed Siddiqui, Nitarshan Rajkumar, Tegan Maharaj +2
Modern machine learning research relies on relatively few carefully curated datasets. Even in these datasets, and typically in `untidy' or raw data, practitioners are faced with si…
Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification
Adriano Lucieri, Fabian Schmeisser, Christoph Peter Balada +3
It is generally believed that the human visual system is biased towards the recognition of shapes rather than textures. This assumption has led to a growing body of work aiming to…
Improving Health Mentioning Classification of Tweets using Contrastive Adversarial Training
Pervaiz Iqbal Khan, Shoaib Ahmed Siddiqui, Imran Razzak +2
Health mentioning classification (HMC) classifies an input text as health mention or not. Figurative and non-health mention of disease words makes the classification task challengi…
Identifying Layers Susceptible to Adversarial Attacks
Shoaib Ahmed Siddiqui, Thomas Breuel
In this paper, we investigate the use of pretraining with adversarial networks, with the objective of discovering the relationship between network depth and robustness. For this pu…
Going Beyond Classification Accuracy Metrics in Model Compression
Vinu Joseph, Shoaib Ahmed Siddiqui, Aditya Bhaskara +5
With the rise in edge-computing devices, there has been an increasing demand to deploy energy and resource-efficient models. A large body of research has been devoted to developing…
Benchmarking adversarial attacks and defenses for time-series data
Shoaib Ahmed Siddiqui, Andreas Dengel, Sheraz Ahmed
The adversarial vulnerability of deep networks has spurred the interest of researchers worldwide. Unsurprisingly, like images, adversarial examples also translate to time-series da…
Two-stage framework for optic disc localization and glaucoma classification in retinal fundus images using deep learning
Muhammad Naseer Bajwa, Muhammad Imran Malik, Shoaib Ahmed Siddiqui +4
With the advancement of powerful image processing and machine learning techniques, CAD has become ever more prevalent in all fields of medicine including ophthalmology. Since optic…
Interpreting Deep Models through the Lens of Data
Dominique Mercier, Shoaib Ahmed Siddiqui, Andreas Dengel +1
Identification of input data points relevant for the classifier (i.e. serve as the support vector) has recently spurred the interest of researchers for both interpretability as wel…
TSInsight: A local-global attribution framework for interpretability in time-series data
Shoaib Ahmed Siddiqui, Dominique Mercier, Andreas Dengel +1
With the rise in the employment of deep learning methods in safety-critical scenarios, interpretability is more essential than ever before. Although many different directions regar…
TSXplain: Demystification of DNN Decisions for Time-Series using Natural Language and Statistical Features
Mohsin Munir, Shoaib Ahmed Siddiqui, Ferdinand Küsters +3
Neural networks (NN) are considered as black-boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environment…
KINN: Incorporating Expert Knowledge in Neural Networks
Muhammad Ali Chattha, Shoaib Ahmed Siddiqui, Muhammad Imran Malik +3
The promise of ANNs to automatically discover and extract useful features/patterns from data without dwelling on domain expertise although seems highly promising but comes at the c…