5 papers
TimeCues Studio: A Workspace for Music Annotation and Algorithm Prototyping
Sapir Caduri, Yoav Goldberg
Multimedia applications require precise music annotation-labeled positions, segments, or loops-placed by hand or algorithmically. Machine-learning algorithms are scalable and effec…
Small Models, Big Results: Achieving Superior Intent Extraction through Decomposition
Danielle Cohen, Yoni Halpern, Noam Kahlon +5
Understanding user intents from UI interaction trajectories remains a challenging, yet crucial, frontier in intelligent agent development. While massive, datacenter-based, multi-mo…
Agent-Initiated Interaction in Phone UI Automation
Noam Kahlon, Guy Rom, Anatoly Efros +6
Phone automation agents aim to autonomously perform a given natural-language user request, such as scheduling appointments or booking a hotel. While much research effort has been d…
Bi-Fact: A Bidirectional Factorization-based Evaluation of Intent Extraction from UI Trajectories
Sapir Caduri, Anatoly Efros, Noam Kahlon +3
Evaluating intent extraction from GUIs demands accurate, fine-grained metrics. This paper introduces Bi-Fact, a novel method that decomposes intents into atomic facts and performs…
Identifying User Goals from UI Trajectories
Omri Berkovitch, Sapir Caduri, Noam Kahlon +3
Identifying underlying user goals and intents has been recognized as valuable in various personalization-oriented settings, such as personalized agents, improved search responses,…