92 citations · 130 across the 21 of their papers we have counts for
25 papers
Recirculation
Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer +2
We describe an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across generation and reasoning tas…
The Topological Trouble With Transformers
Michael C. Mozer, Shoaib Ahmed Siddiqui, Rosanne Liu
Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tr…
Position: Capability Control Should be a Separate Goal From Alignment
Shoaib Ahmed Siddiqui, Eleni Triantafillou, David Krueger +1
Foundation models are trained on broad data distributions, yielding generalist capabilities that enable many downstream applications but also expand the space of potential misuse a…
From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization
Shoaib Ahmed Siddiqui, Adrian Weller, David Krueger +3
Recent unlearning methods for LLMs are vulnerable to relearning attacks: knowledge believed-to-be-unlearned re-emerges by fine-tuning on a small set of (even seemingly-unrelated) e…
JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs
Aliaksandra Shysheya, John Bronskill, James Requeima +4
We introduce a simple method for probabilistic predictions on tabular data based on Large Language Models (LLMs) called JoLT (Joint LLM Process for Tabular data). JoLT uses the in-…
Exploring the design space of deep-learning-based weather forecasting systems
Shoaib Ahmed Siddiqui, Jean Kossaifi, Boris Bonev +4
Despite tremendous progress in developing deep-learning-based weather forecasting systems, their design space, including the impact of different design choices, is yet to be well u…