127 citations · 174 across the 25 of their papers we have counts for
27 papers
Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Families
Abdulrahman Alabdulkareem, Irene Tenison, Yang Zhang +3
LLM multi-agent systems split work across agents that communicate through text. An agent summarizes its findings, another agent receives the text. This constrains communication to…
The Sparsity Whisperer
Linghao Kong, Inimai Subramanian, Micah Adler +3
Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs. We argue that this overlooks a…
LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface
Michael Hind, Basel Shbita, Bo Wu +5
Textual Large Language Models (LLMs) provide a simple and familiar interface: a string of text is used for both input and output. However, the information conveyed to an LLM often…
How Much Reasoning Do Retrieval-Augmented Models Add beyond LLMs? A Benchmarking Framework for Multi-Hop Inference over Hybrid Knowledge
Junhong Lin, Bing Zhang, Song Wang +4
Large language models (LLMs) continue to struggle with knowledge-intensive questions that require up-to-date information and multi-hop reasoning. Augmenting LLMs with hybrid extern…
BOAD: Discovering Hierarchical Software Engineering Agents via Bandit Optimization
Iris Xu, Guangtao Zeng, Zexue He +5
Large language models (LLMs) have shown strong reasoning and coding capabilities, yet they struggle to generalize to real-world software engineering (SWE) problems that are long-ho…
ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generation
Jovana Kondic, Pengyuan Li, Dhiraj Joshi +12
Chart-to-code reconstruction -- the task of recovering executable plotting scripts from chart images -- provides important insights into a model's ability to ground data visualizat…
Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering
Guangtao Zeng, Maohao Shen, Delin Chen +8
Language models (LMs) perform well on standardized coding benchmarks but struggle with real-world software engineering tasks such as resolving GitHub issues in SWE-Bench, especiall…
Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence
Granite Vision Team, Leonid Karlinsky, Assaf Arbelle +60
We introduce Granite Vision, a lightweight large language model with vision capabilities, specifically designed to excel in enterprise use cases, particularly in visual document un…
M+: Extending MemoryLLM with Scalable Long-Term Memory
Yu Wang, Dmitry Krotov, Yuanzhe Hu +6
Equipping large language models (LLMs) with latent-space memory has attracted increasing attention as they can extend the context window of existing language models. However, retai…
LInK: Learning Joint Representations of Design and Performance Spaces through Contrastive Learning for Mechanism Synthesis
Amin Heyrani Nobari, Akash Srivastava, Dan Gutfreund +2
In this paper, we introduce LInK, a novel framework that integrates contrastive learning of performance and design space with optimization techniques for solving complex inverse pr…
Constraining Generative Models for Engineering Design with Negative Data
Lyle Regenwetter, Giorgio Giannone, Akash Srivastava +2
Generative models have recently achieved remarkable success and widespread adoption in society, yet they often struggle to generate realistic and accurate outputs. This challenge e…
3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation
Guangyao Zhou, Nishad Gothoskar, Lirui Wang +5
The ability to perceive and understand 3D scenes is crucial for many applications in computer vision and robotics. Inverse graphics is an appealing approach to 3D scene understandi…
Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design
Lyle Regenwetter, Akash Srivastava, Dan Gutfreund +1
Deep generative models such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), Diffusion Models, and Transformers, have shown great promise in a variety of…
LINKS: A dataset of a hundred million planar linkage mechanisms for data-driven kinematic design
Amin Heyrani Nobari, Akash Srivastava, Dan Gutfreund +1
In this paper, we introduce LINKS, a dataset of 100 million one degree of freedom planar linkage mechanisms and 1.1 billion coupler curves, which is more than 1000 times larger tha…
Solving the Baby Intuitions Benchmark with a Hierarchically Bayesian Theory of Mind
Tan Zhi-Xuan, Nishad Gothoskar, Falk Pollok +3
To facilitate the development of new models to bridge the gap between machine and human social intelligence, the recently proposed Baby Intuitions Benchmark (arXiv:2102.11938) prov…
Finding Fallen Objects Via Asynchronous Audio-Visual Integration
Chuang Gan, Yi Gu, Siyuan Zhou +7
The way an object looks and sounds provide complementary reflections of its physical properties. In many settings cues from vision and audition arrive asynchronously but must be in…
3DP3: 3D Scene Perception via Probabilistic Programming
Nishad Gothoskar, Marco Cusumano-Towner, Ben Zinberg +6
We present 3DP3, a framework for inverse graphics that uses inference in a structured generative model of objects, scenes, and images. 3DP3 uses (i) voxel models to represent the 3…
Incorporating Rich Social Interactions Into MDPs
Ravi Tejwani, Yen-Ling Kuo, Tianmin Shu +5
Much of what we do as humans is engage socially with other agents, a skill that robots must also eventually possess. We demonstrate that a rich theory of social interactions origin…
The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AI
Chuang Gan, Siyuan Zhou, Jeremy Schwartz +8
We introduce a visually-guided and physics-driven task-and-motion planning benchmark, which we call the ThreeDWorld Transport Challenge. In this challenge, an embodied agent equipp…
AGENT: A Benchmark for Core Psychological Reasoning
Tianmin Shu, Abhishek Bhandwaldar, Chuang Gan +6
For machine agents to successfully interact with humans in real-world settings, they will need to develop an understanding of human mental life. Intuitive psychology, the ability t…
Template Controllable keywords-to-text Generation
Abhijit Mishra, Md Faisal Mahbub Chowdhury, Sagar Manohar +2
This paper proposes a novel neural model for the understudied task of generating text from keywords. The model takes as input a set of un-ordered keywords, and part-of-speech (POS)…
Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling
Akash Srivastava, Yamini Bansal, Yukun Ding +15
Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the (aggregate) posterior to encourage statistical independence of the…
ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
Chuang Gan, Jeremy Schwartz, Seth Alter +21
We introduce ThreeDWorld (TDW), a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between m…
SimVAE: Simulator-Assisted Training forInterpretable Generative Models
Akash Srivastava, Jessie Rosenberg, Dan Gutfreund +1
This paper presents a simulator-assisted training method (SimVAE) for variational autoencoders (VAE) that leads to a disentangled and interpretable latent space. Training SimVAE is…