4 papers
Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models
Ross Tieman, Evan Markou
Diversity is a widely observed factor in the resilient function of collective systems, yet the type of diversity that matters depends on the properties and failure modes of the sys…
Towards Scalable Backpropagation-Free Gradient Estimation
Daniel Wang, Evan Markou, Dylan Campbell
While backpropagation--reverse-mode automatic differentiation--has been extraordinarily successful in deep learning, it requires two passes (forward and backward) through the neura…
Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings
Evan Markou, Thalaiyasingam Ajanthan, Stephen Gould
Many high-dimensional optimisation problems exhibit rich geometric structures in their set of minimisers, often forming smooth manifolds due to over-parametrisation or symmetries.…
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame
Evan Markou, Thalaiyasingam Ajanthan, Stephen Gould
Neural Collapse (NC) is a recently observed phenomenon in neural networks that characterises the solution space of the final classifier layer when trained until zero training loss.…