7 papers
Computer-Using World Model
Yiming Guan, Rui Yu, John Zhang +15
Agents operating in complex software environments benefit from reasoning about the consequences of their actions, as even a single incorrect user interface (UI) operation can derai…
Mano Technical Report
Tianyu Fu, Anyang Su, Chenxu Zhao +20
Graphical user interfaces (GUIs) are the primary medium for human-computer interaction, yet automating GUI interactions remains challenging due to the complexity of visual elements…
BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining
Jie Hao, Rui Yu, Wei Zhang +3
Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks. However, existing approac…
Reinforcement Learning enhanced Online Adaptive Clinical Decision Support via Digital Twin powered Policy and Treatment Effect optimized Reward
Xinyu Qin, Ruiheng Yu, Lu Wang
Clinical decision support must adapt online under safety constraints. We present an online adaptive tool where reinforcement learning provides the policy, a patient digital twin pr…
FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making
Yucen Wang, Rui Yu, Shenghua Wan +2
Foundation Models (FMs) and World Models (WMs) offer complementary strengths in task generalization at different levels. In this work, we propose FOUNDER, a framework that integrat…
Reward Models in Deep Reinforcement Learning: A Survey
Rui Yu, Shenghua Wan, Yucen Wang +4
In reinforcement learning (RL), agents continually interact with the environment and use the feedback to refine their behavior. To guide policy optimization, reward models are intr…
JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models
Jiaxin Song, Yixu Wang, Jie Li +4
Vision-Language Models (VLMs) exhibit impressive performance, yet the integration of powerful vision encoders has significantly broadened their attack surface, rendering them incre…