7 citations · 14 across the 15 of their papers we have counts for
16 papers
Tracing distinguishability through transformer processing with stochastic LayerNorm
Kieran Murphy
Representational similarity is foundational to analyses of deep networks, yet distances between point-valued representations are not intrinsically tied to downstream function: near…
Attribution via Distributional Paths for Information Revelation
Kieran A. Murphy, Shameen Shrestha
Feature attribution methods explain predictions by assigning importance scores to input features. Path-based methods such as Integrated Gradients are especially appealing because t…
InfoChess: A Game of Adversarial Inference and a Laboratory for Quantifiable Information Control
Kieran A. Murphy
We propose InfoChess, a symmetric adversarial game that elevates competitive information acquisition to the primary objective. There is no piece capture, removing material incentiv…
From independent patches to coordinated attention: Controlling information flow in vision transformers
Kieran A. Murphy
We make the information transmitted by attention an explicit, measurable quantity in vision transformers. By inserting variational information bottlenecks on all attention-mediated…
Which bits went where? Past and future transfer entropy decomposition with the information bottleneck
Kieran A. Murphy, Zhuowen Yin, Dani S. Bassett
Whether the system under study is a shoal of fish, a collection of neurons, or a set of interacting atmospheric and oceanic processes, transfer entropy measures the flow of informa…
Surveying the space of descriptions of a composite system with machine learning
Kieran A. Murphy, Yujing Zhang, Dani S. Bassett
Multivariate information theory provides a general and principled framework for understanding how the components of a complex system are connected. Existing analyses are coarse in…
Comparing the information content of probabilistic representation spaces
Kieran A. Murphy, Sam Dillavou, Dani S. Bassett
Probabilistic representation spaces convey information about a dataset and are shaped by factors such as the training data, network architecture, and loss function. Comparing the i…
Mechanical prions: Self-assembling microstructures
Mathieu Ouellet, Dani S. Bassett, Lee C. Bassett +2
Prions are misfolded proteins that transmit their structural arrangement to neighboring proteins. In biological systems, prion dynamics can produce a variety of complex functional…
Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck
Kieran A. Murphy, Dani S. Bassett
Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution wit…
Intrinsically motivated graph exploration using network theories of human curiosity
Shubhankar P. Patankar, Mathieu Ouellet, Juan Cervino +3
Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a grap…
Information decomposition in complex systems via machine learning
Kieran A. Murphy, Dani S. Bassett
One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscop…
Interpretability with full complexity by constraining feature information
Kieran A. Murphy, Dani S. Bassett
Interpretability is a pressing issue for machine learning. Common approaches to interpretable machine learning constrain interactions between features of the input, rendering the e…
Characterizing information loss in a chaotic double pendulum with the Information Bottleneck
Kieran A. Murphy, Dani S. Bassett
A hallmark of chaotic dynamics is the loss of information with time. Although information loss is often expressed through a connection to Lyapunov exponents -- valid in the limit o…
The Distributed Information Bottleneck reveals the explanatory structure of complex systems
Kieran A. Murphy, Dani S. Bassett
The fruits of science are relationships made comprehensible, often by way of approximation. While deep learning is an extremely powerful way to find relationships in data, its use…
Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold
Kieran Murphy, Carlos Esteves, Varun Jampani +2
Single image pose estimation is a fundamental problem in many vision and robotics tasks, and existing deep learning approaches suffer by not completely modeling and handling: i) un…
Learning ABCs: Approximate Bijective Correspondence for isolating factors of variation with weak supervision
Kieran A. Murphy, Varun Jampani, Srikumar Ramalingam +1
Representational learning forms the backbone of most deep learning applications, and the value of a learned representation is intimately tied to its information content regarding d…