2 papers
cs.LG2025
Impact of Layer Norm on Memorization and Generalization in Transformers
Rishi Singhal, Jung-Eun Kim
Layer Normalization (LayerNorm) is one of the fundamental components in transformers that stabilizes training and improves optimization. In recent times, Pre-LayerNorm transformers…
cs.CV2024
Extracting Usable Predictions from Quantized Networks through Uncertainty Quantification for OOD Detection
Rishi Singhal, Srinath Srinivasan
OOD detection has become more pertinent with advances in network design and increased task complexity. Identifying which parts of the data a given network is misclassifying has bec…