4 papers · 1 filter
MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents
Kaixin Ma, Di Feng, Alexander Metz +3
We introduce MM-ToolSandBox, a benchmark and evaluation framework for visually grounded tool-calling agents. The framework provides a stateful execution environment spanning 500+ t…
SO-Bench: A Structural Output Evaluation of Multimodal LLMs
Di Feng, Kaixin Ma, Feng Nan +9
Multimodal large language models (MLLMs) are increasingly deployed in real-world, agentic settings where outputs must not only be correct, but also conform to predefined data schem…
Towards Multimodal Understanding via Stable Diffusion as a Task-Aware Feature Extractor
Vatsal Agarwal, Matthew Gwilliam, Gefen Kohavi +3
Recent advances in multimodal large language models (MLLMs) have enabled image-based question-answering capabilities. However, a key limitation is the use of CLIP as the visual enc…
Revealing the Utilized Rank of Subspaces of Learning in Neural Networks
Isha Garg, Christian Koguchi, Eshan Verma +1
In this work, we study how well the learned weights of a neural network utilize the space available to them. This notion is related to capacity, but additionally incorporates the i…