5 papers
LUCid: Redefining Relevance For Lifelong Personalization
Chimaobi Okite, Anika Misra, Joyce Chai +1
Work to date has mainly relied on semantic proximity to identify relevant content for lifelong personalization. However, situational relevance is often more important for determini…
SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
Josue Torres-Fonseca, Naihao Deng, Yinpei Dai +5
Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insuffic…
Evaluating Long-Context Reasoning in LLM-Based WebAgents
Andy Chung, Yichi Zhang, Kaixiang Lin +3
As large language model (LLM)-based agents become increasingly integrated into daily digital interactions, their ability to reason across long interaction histories becomes crucial…
Proactive Assistant Dialogue Generation from Streaming Egocentric Videos
Yichi Zhang, Xin Luna Dong, Zhaojiang Lin +5
Recent advances in conversational AI have been substantial, but developing real-time systems for perceptual task guidance remains challenging. These systems must provide interactiv…
Temporally-Grounded Language Generation: A Benchmark for Real-Time Vision-Language Models
Keunwoo Peter Yu, Joyce Chai
Vision-language models (VLMs) have shown remarkable progress in offline tasks such as image captioning and video question answering. However, real-time interactive environments imp…