2 citations · 2 across the 4 of their papers we have counts for
6 papers
LinQuant: Functional Invariances for Low-Precision Linear Recurrences
Neehal Tumma, Philipp Nazari, Rohan Cherukuri +2
Linear recurrences are emerging as practical alternatives to softmax attention in large language models, appearing in both standalone and hybrid architectures. However, despite the…
Depth Memory
David González-Martínez, Philipp Nazari, T. Konstantin Rusch +1
Residual connections propagate information through a network by additively accumulating layer outputs. Recently proposed alternatives, such as Manifold-Constrained Hyper-Connection…
On State Reduction in Linear Attention
Philipp Nazari, T. Konstantin Rusch
Linear attention offers a computationally efficient yet expressive alternative to softmax attention. However, recent empirical results indicate that the hidden state of trained lin…
The Curious Case of In-Training Compression of State Space Models
Makram Chahine, Philipp Nazari, Daniela Rus +1
State Space Models (SSMs), developed to tackle long sequence modeling tasks efficiently, offer both parallelizable training and fast inference. At their core are recurrent dynamica…
Entropy Aware Message Passing in Graph Neural Networks
Philipp Nazari, Oliver Lemke, Davide Guidobene +1
Deep Graph Neural Networks struggle with oversmoothing. This paper introduces a novel, physics-inspired GNN model designed to mitigate this issue. Our approach integrates with exis…
Geometric Autoencoders -- What You See is What You Decode
Philipp Nazari, Sebastian Damrich, Fred A. Hamprecht
Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can…