4 papers · 1 filter
Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
Luca Baroni, Galvin Khara, Joachim Schaeffer +2
Layer-wise normalization (LN) is an essential component of virtually all transformer-based large language models. While its effects on training stability are well documented, its r…
Interpretation of High-Dimensional Regression Coefficients by Comparison with Linearized Compressing Features
Joachim Schaeffer, Jinwook Rhyu, Robin Droop +2
Linear regression is often deemed inherently interpretable; however, challenges arise for high-dimensional data. We focus on further understanding how linear regression approximate…
Gaussian process-based online health monitoring and fault analysis of lithium-ion battery systems from field data
Joachim Schaeffer, Eric Lenz, Duncan Gulla +3
Health monitoring, fault analysis, and detection are critical for the safe and sustainable operation of battery systems. We apply Gaussian process resistance models on lithium iron…
Systematic Feature Design for Cycle Life Prediction of Lithium-Ion Batteries During Formation
Jinwook Rhyu, Joachim Schaeffer, Michael L. Li +4
Optimization of the formation step in lithium-ion battery manufacturing is challenging due to limited physical understanding of solid electrolyte interphase formation and the long…