4 papers
SVCBench: A Streaming Video Counting Benchmark for Spatial-Temporal State Maintenance
Pengyiang Liu, Zhongyue Shi, Hongye Hao +7
Video understanding requires models to continuously track and update world state during playback. Although existing benchmarks have advanced video understanding evaluation across m…
PromptWise: Online Learning for Cost-Aware Prompt Assignment in Generative Models
Xiaoyan Hu, Lauren Pick, Ho-fung Leung +1
The rapid advancement of generative AI has provided users with a wide range of well-trained models to address diverse prompts. When selecting a model for a given prompt, users shou…
PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMs
Xiaoyan Hu, Ho-fung Leung, Farzan Farnia
Selecting a sample generation scheme from multiple prompt-based generative models, including large language models (LLMs) and prompt-guided image and video generation models, is ty…
A Multi-Armed Bandit Approach to Online Selection and Evaluation of Generative Models
Xiaoyan Hu, Ho-fung Leung, Farzan Farnia
Existing frameworks for evaluating and comparing generative models consider an offline setting, where the evaluator has access to large batches of data produced by the models. Howe…