StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents

Yan Yang, Xiangru Jian, Ziyang Luo +7

Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering…