73 citations · 91 across the 28 of their papers we have counts for
31 papers
\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
Jianghui Wang, Silong Yong, Francesco Orabona +3
Low-Rank Adaptation (LoRA) has become a widely adopted technique for efficient neural network fine-tuning, decomposing model updates into low-rank matrices. However, LoRA remains c…
Theory of Mind Guided Strategy Adaptation for Zero-Shot Coordination
Andrew Ni, Simon Stepputtis, Stefanos Nikolaidis +3
A central challenge in multi-agent reinforcement learning is enabling agents to adapt to previously unseen teammates in a zero-shot fashion. Prior work in zero-shot coordination of…
Adaptively Coordinating with Novel Partners via Learned Latent Strategies
Benjamin Li, Shuyang Shi, Lucia Romero +7
Adaptation is the cornerstone of effective collaboration among heterogeneous team members. In human-agent teams, artificial agents need to adapt to their human partners in real tim…
Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration
Benjamin Li, Shuyang Shi, Lucia Romero +7
In collaborative tasks, being able to adapt to your teammates is a necessary requirement for success. When teammates are heterogeneous, such as in human-agent teams, agents need to…
ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
Zifu Wan, Ce Zhang, Silong Yong +6
Recent Large Vision-Language Models (LVLMs) have introduced a new paradigm for understanding and reasoning about image input through textual responses. Although they have achieved…
Energy-Based Transfer for Reinforcement Learning
Zeyun Deng, Jasorsi Ghosh, Fiona Xie +4
Reinforcement learning remains sample inefficient, particularly when agents must adapt over time to related but distinct tasks, as in continual learning settings. Efficiency can be…
InstructPart: Task-Oriented Part Segmentation with Instruction Reasoning
Zifu Wan, Yaqi Xie, Ce Zhang +5
Large multimodal foundation models, particularly in the domains of language and vision, have significantly advanced various tasks, including robotics, autonomous driving, informati…
Spectral-Aware Global Fusion for RGB-Thermal Semantic Segmentation
Ce Zhang, Zifu Wan, Simon Stepputtis +2
Semantic segmentation relying solely on RGB data often struggles in challenging conditions such as low illumination and obscured views, limiting its reliability in critical applica…
VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models
Ce Zhang, Kaixin Ma, Tianqing Fang +5
Recent Large Vision-Language Models (LVLMs) have advanced multi-modal understanding by incorporating finer-grained visual perception and encoding. However, such methods incur signi…
Model-Agnostic Policy Explanations with Large Language Models
Zhang Xi-Jia, Yue Guo, Shufei Chen +4
Intelligent agents, such as robots, are increasingly deployed in real-world, human-centric environments. To foster appropriate human trust and meet legal and ethical standards, the…
OMG: Opacity Matters in Material Modeling with Gaussian Splatting
Silong Yong, Venkata Nagarjun Pudureddiyur Manivannan, Bernhard Kerbl +4
Decomposing geometry, materials and lighting from a set of images, namely inverse rendering, has been a long-standing problem in computer vision and graphics. Recent advances in ne…
Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning
Muhan Lin, Shuyang Shi, Yue Guo +7
Credit assignment, the process of attributing credit or blame to individual agents for their contributions to a team's success or failure, remains a fundamental challenge in multi-…
Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models
Ce Zhang, Zifu Wan, Zhehan Kan +7
While recent Large Vision-Language Models (LVLMs) have shown remarkable performance in multi-modal tasks, they are prone to generating hallucinatory text responses that do not alig…
HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation
Zirui Wang, Xinran Zhao, Simon Stepputtis +4
Understanding and predicting human actions has been a long-standing challenge and is a crucial measure of perception in robotics AI. While significant progress has been made in ant…
Symbolic Graph Inference for Compound Scene Understanding
FNU Aryan, Simon Stepputtis, Sarthak Bhagat +4
Scene understanding is a fundamental capability needed in many domains, ranging from question-answering to robotics. Unlike recent end-to-end approaches that must explicitly learn…
Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Models
Muhan Lin, Shuyang Shi, Yue Guo +6
The correct specification of reward models is a well-known challenge in reinforcement learning. Hand-crafted reward functions often lead to inefficient or suboptimal policies and m…
GL-NeRF: Gauss-Laguerre Quadrature Enables Training-Free NeRF Acceleration
Silong Yong, Yaqi Xie, Simon Stepputtis +1
Volume rendering in neural radiance fields is inherently time-consuming due to the large number of MLP calls on the points sampled per ray. Previous works would address this issue…
Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models
Ce Zhang, Simon Stepputtis, Katia Sycara +1
Test-time adaptation, which enables models to generalize to diverse data with unlabeled test samples, holds significant value in real-world scenarios. Recently, researchers have ap…
Multi-Agent Transfer Learning via Temporal Contrastive Learning
Weihao Zeng, Joseph Campbell, Simon Stepputtis +1
This paper introduces a novel transfer learning framework for deep multi-agent reinforcement learning. The approach automatically combines goal-conditioned policies with temporal c…
Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
Zifu Wan, Pingping Zhang, Yuhao Wang +4
Multi-modal semantic segmentation significantly enhances AI agents' perception and scene understanding, especially under adverse conditions like low-light or overexposed environmen…
ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decomposition
Samuel Li, Sarthak Bhagat, Joseph Campbell +4
Task-oriented grasping of unfamiliar objects is a necessary skill for robots in dynamic in-home environments. Inspired by the human capability to grasp such objects through intuiti…
CBGT-Net: A Neuromimetic Architecture for Robust Classification of Streaming Data
Shreya Sharma, Dana Hughes, Katia Sycara
This paper describes CBGT-Net, a neural network model inspired by the cortico-basal ganglia-thalamic (CBGT) circuits found in mammalian brains. Unlike traditional neural network mo…
HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation
Ce Zhang, Simon Stepputtis, Joseph Campbell +2
Being able to understand visual scenes is a precursor for many downstream tasks, including autonomous driving, robotics, and other vision-based approaches. A common approach enabli…
Enhancing Vision-Language Few-Shot Adaptation with Negative Learning
Ce Zhang, Simon Stepputtis, Katia Sycara +1
Large-scale pre-trained Vision-Language Models (VLMs) have exhibited impressive zero-shot performance and transferability, allowing them to adapt to downstream tasks in a data-effi…