4 papers
Beyond Omni: A Unified MLLM for Embodied Sensory Understanding
Shiyun Xiong, Dongming Wu, Peiwen Sun +5
Recent embodied systems increasingly build on multimodal large language models (MLLMs) for their pretrained understanding and reasoning capabilities, but their perception of the ph…
Benchmark Everything Everywhere All at Once
Shiyun Xiong, Dongming Wu, Peiwen Sun +5
Benchmarks are fundamental for evaluating and advancing LLMs and MLLMs by providing standardized and explicit measures of performance. However, their construction is labor-intensiv…
BehaviorSFT: Behavioral Token Conditioning for Clinical Agents Across the Proactivity Spectrum
Yubin Kim, Zhiyuan Hu, Hyewon Jeong +11
Large Language Models (LLMs) as clinical agents require careful behavioral adaptation. While adept at reactive tasks (e.g., diagnosis reasoning), LLMs often struggle with proactive…
Guiding VLM Agents with Process Rewards at Inference Time for GUI Navigation
Zhiyuan Hu, Shiyun Xiong, Yifan Zhang +5
Recent advancements in visual language models (VLMs) have notably enhanced their capabilities in handling complex Graphical User Interface (GUI) interaction tasks. Despite these im…