14 citations · 31 across the 22 of their papers we have counts for
27 papers
Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Families
Abdulrahman Alabdulkareem, Irene Tenison, Yang Zhang +3
LLM multi-agent systems split work across agents that communicate through text. An agent summarizes its findings, another agent receives the text. This constrains communication to…
ArtifactArena: Evaluating Models by What They Build in the Physical World
Kushagra Tiwary*, David Mayo*, Nikhil Behari +5
To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce \textsc{ArtifactArena…
PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents
Vighnesh Subramaniam, Boris Katz, Brian Cheung +3
Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts,…
Grounding Social Perception in Intuitive Physics
Lance Ying, Aydan Y. Huang, Aviv Netanyahu +4
People infer rich social information from others' actions. These inferences are often constrained by the physical world: what agents can do, what obstacles permit, and how the phys…
Network of Theseus (like the ship)
Vighnesh Subramaniam, Colin Conwell, Boris Katz +2
A standard assumption in deep learning is that the inductive bias introduced by a neural network architecture must persist from training through inference. The architecture you tra…
Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimuli
Andrii Zahorodnii, Christopher Wang, Geeling Chau +7
High-resolution neural datasets enable foundation models for the next generation of brain-computer interfaces and neurological treatments. The community requires rigorous benchmark…
Brain Treebank: Large-scale intracranial recordings from naturalistic language stimuli
Christopher Wang, Adam Uri Yaari, Aaditya K Singh +10
We present the Brain Treebank, a large-scale dataset of electrophysiological neural responses, recorded from intracranial probes while 10 subjects watched one or more Hollywood mov…
BrainBits: How Much of the Brain are Generative Reconstruction Methods Using?
David Mayo, Christopher Wang, Asa Harbin +4
When evaluating stimuli reconstruction results it is tempting to assume that higher fidelity text and image generation is due to an improved understanding of the brain or more powe…
Training the Untrainable: Introducing Inductive Bias via Representational Alignment
Vighnesh Subramaniam, David Mayo, Colin Conwell +4
We demonstrate that architectures which traditionally are considered to be ill-suited for a task can be trained using inductive biases from another architecture. We call a network…
Revealing Vision-Language Integration in the Brain with Multimodal Networks
Vighnesh Subramaniam, Colin Conwell, Christopher Wang +4
We use (multi)modal deep neural networks (DNNs) to probe for sites of multimodal integration in the human brain by predicting stereoencephalography (SEEG) recordings taken while hu…
SecureLLM: Using Compositionality to Build Provably Secure Language Models for Private, Sensitive, and Secret Data
Abdulrahman Alabdulkareem, Christian M Arnold, Yerim Lee +3
Traditional security mechanisms isolate resources from users who should not access them. We reflect the compositional nature of such security mechanisms back into the structure of…
Population Transformer: Learning Population-level Representations of Neural Activity
Geeling Chau, Christopher Wang, Sabera Talukder +5
We present a self-supervised framework that learns population-level codes for arbitrary ensembles of neural recordings at scale. We address key challenges in scaling models with ne…
BrainBERT: Self-supervised representation learning for intracranial recordings
Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari +4
We create a reusable Transformer, BrainBERT, for intracranial recordings bringing modern representation learning approaches to neuroscience. Much like in NLP and speech recognition…
Query The Agent: Improving sample efficiency through epistemic uncertainty estimation
Julian Alverio, Boris Katz, Andrei Barbu
Curricula for goal-conditioned reinforcement learning agents typically rely on poor estimates of the agent's epistemic uncertainty or fail to consider the agents' epistemic uncerta…
Developing a Series of AI Challenges for the United States Department of the Air Force
Vijay Gadepally, Gregory Angelides, Andrei Barbu +39
Through a series of federal initiatives and orders, the U.S. Government has been making a concerted effort to ensure American leadership in AI. These broad strategy documents have…
Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset
Ian Palmer, Andrew Rouditchenko, Andrei Barbu +2
Visually-grounded spoken language datasets can enable models to learn cross-modal correspondences with very weak supervision. However, modern audio-visual datasets contain biases t…
Incorporating Rich Social Interactions Into MDPs
Ravi Tejwani, Yen-Ling Kuo, Tianmin Shu +5
Much of what we do as humans is engage socially with other agents, a skill that robots must also eventually possess. We demonstrate that a rich theory of social interactions origin…
Trajectory Prediction with Linguistic Representations
Yen-Ling Kuo, Xin Huang, Andrei Barbu +4
Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions. We present a novel trajectory predictio…
PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
Aviv Netanyahu, Tianmin Shu, Boris Katz +2
The ability to perceive and reason about social interactions in the context of physical environments is core to human social intelligence and human-machine cooperation. However, no…
Migratable AI: Personalizing Dialog Conversations with migration context
Ravi Tejwani, Boris Katz, Cynthia Breazeal
The migration of conversational AI agents across different embodiments in order to maintain the continuity of the task has been recently explored to further improve user experience…
Learning a natural-language to LTL executable semantic parser for grounded robotics
Christopher Wang, Candace Ross, Yen-Ling Kuo +2
Children acquire their native language with apparent ease by observing how language is used in context and attempting to use it themselves. They do so without laborious annotations…
Compositional Networks Enable Systematic Generalization for Grounded Language Understanding
Yen-Ling Kuo, Boris Katz, Andrei Barbu
Humans are remarkably flexible when understanding new sentences that include combinations of concepts they have never encountered before. Recent work has shown that while deep netw…
Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas
Yen-Ling Kuo, Boris Katz, Andrei Barbu
We demonstrate a reinforcement learning agent which uses a compositional recurrent neural network that takes as input an LTL formula and determines satisfying actions. The input LT…
Investigating the Decoders of Maximum Likelihood Sequence Models: A Look-ahead Approach
Yu-Siang Wang, Yen-Ling Kuo, Boris Katz
We demonstrate how we can practically incorporate multi-step future information into a decoder of maximum likelihood sequence models. We propose a "k-step look-ahead" module to con…