265 citations · 1k across the 84 of their papers we have counts for
86 papers
Efficient DP-SGD for LLMs with Randomized Clipping
Enayat Ullah, Sai Aparna Aketi, Devansh Gupta +2
Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Differential privacy (DP), the de facto standard for formal privacy guarantees, pr…
Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs
Amin Banayeeanzade, Qingchuan Yang, Dhruv Tarsadiya +6
Diversity is essential for language-model applications ranging from creative generation to scientific discovery, yet modern LLMs often collapse into a narrow subset of plausible ou…
Early Stopping for Large Reasoning Models via Confidence Dynamics
Parsa Hosseini, Sumit Nawathe, Mahdi Salmani +2
Large reasoning models rely on long chain-of-thought generation to solve complex problems, but extended reasoning often incurs substantial computational cost and can even degrade p…
Memory Caching: RNNs with Growing Memory
Ali Behrouz, Zeman Li, Yuan Deng +3
Transformers have been established as the de-facto backbones for most recent advances in sequence modeling, mainly due to their growing memory capacity that scales with the context…
Nested Learning: The Illusion of Deep Learning Architectures
Ali Behrouz, Meisam Razaviyayn, Peilin Zhong +1
Despite the recent progresses, particularly in developing Language Models, there are fundamental challenges and unanswered questions about how such models can continually learn/mem…
TNT: Improving Chunkwise Training for Test-Time Memorization
Zeman Li, Ali Behrouz, Yuan Deng +5
Recurrent neural networks (RNNs) with deep test-time memorization modules, such as Titans and TTT, represent a promising, linearly-scaling paradigm distinct from Transformers. Whil…