11 papers
The Role of Feed-Forward Layers in Transformer Dynamics
Thomas Jacob Maranzatto, Semih Akkoc, Sennur Ulukus
We study the dynamical behavior of tokens in transformers from a control-theoretic perspective. Our model includes the feed-forward layer present after the self-attention mechanism…
On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers
Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5
We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems. In this perspective, tokens are modeled as particles…
Deriving Moments in the Age of Gossip Process from Percolation
Thomas Jacob Maranzatto, Sennur Ulukus
This paper concerns fundamental identities in the study of age of information (AoI) in gossip networks. We recover known recursive identities for arbitrary kth moments of the age p…
New Capacity Bounds for PIR on Graph and Multigraph-Based Replicated Storage
Xiangliang Kong, Shreya Meel, Thomas Jacob Maranzatto +2
In this paper, we study the problem of private information retrieval (PIR) in both graph-based and multigraph-based replication systems, where each file is stored on exactly two se…
Information Freshness in Dynamic Gossip Networks
Arunabh Srivastava, Thomas Jacob Maranzatto, Sennur Ulukus
We consider a source that shares updates with a network of gossiping nodes. The network's topology switches between two arbitrary topologies, with switching governed by a two-s…
Information Degradation and Misinformation in Gossip Networks
Thomas Jacob Maranzatto, Arunabh Srivastava, Sennur Ulukus
We study networks of gossiping users where a source observing a process sends updates to an underlying graph. Nodes in the graph update their neighbors randomly and nodes always ac…
Private Information Retrieval on Multigraph-Based Replicated Storage
Shreya Meel, Xiangliang Kong, Thomas Jacob Maranzatto +2
We consider the private information retrieval (PIR) problem for a multigraph-based replication system, where each set of files is stored on two of the servers according to an u…
Age of Gossip With Time-Varying Topologies
Arunabh Srivastava, Thomas Jacob Maranzatto, Sennur Ulukus
We consider a gossiping network, where a source node sends updates to a network of gossiping nodes. Meanwhile, the connectivity topology of the gossiping network changes over t…
Age of Gossip with the Push-Pull Protocol
Arunabh Srivastava, Thomas Jacob Maranzatto, Sennur Ulukus
We consider a wireless network where a source generates packets and forwards them to a network containing nodes. The nodes in the network use the asynchronous push, pull or pus…
Age of gossip from connective properties via first passage percolation
Thomas Jacob Maranzatto, Marcus Michelen
In gossip networks, a source node forwards time-stamped updates to a network of observers according to a Poisson process. The observers then update each other on this information a…
Age of Gossip in Random and Bipartite Networks
Thomas Jacob Maranzatto
In this paper we study gossip networks where a source observing a process sends updates to an underlying graph. Nodes in the graph communicate to their neighbors by randomly sendin…