4 papers
Impact of Layer Norm on Memorization and Generalization in Transformers
Rishi Singhal, Jung-Eun Kim
Layer Normalization (LayerNorm) is one of the fundamental components in transformers that stabilizes training and improves optimization. In recent times, Pre-LayerNorm transformers…
Explaining How Quantization Disparately Skews a Model
Abhimanyu Bellam, Jung-Eun Kim
Post Training Quantization (PTQ) is widely adopted due to its high compression capacity and speed with minimal impact on accuracy. However, we observed that disparate impacts are e…
Severing Spurious Correlations with Data Pruning
Varun Mulchandani, Jung-Eun Kim
Deep neural networks have been shown to learn and rely on spurious correlations present in the data that they are trained on. Reliance on such correlations can cause these networks…
Trustworthy AI: Safety, Bias, and Privacy -- A Survey
Xingli Fang, Jianwei Li, Varun Mulchandani +1
The capabilities of artificial intelligence systems have been advancing to a great extent, but these systems still struggle with failure modes, vulnerabilities, and biases. In this…