2 citations · 5 across the 15 of their papers we have counts for
18 papers
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
Lance Ying, Jinzhou Wu, Yingshan Susan Wang +53
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways…
Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States
Zixuan Wang, Yufan Zhou, Jinzhou Tang +16
As language models become more capable, long-term collaboration in learning, reasoning, and decision-making calls for a deeper understanding of the people they serve. Yet training…
HyperTrace: Hypothesis-Based Preference Tracing for Online LLM Personalization
Jianzhi Shen, Keyu Mao, Minghao Shao +7
Personalized language models aim to adapt responses to individual users, whose preferences are often latent and revealed gradually through interaction. Existing training-free metho…
MindZero: Learning Online Mental Reasoning With Zero Annotations
Shunchi Zhang, Jin Lu, Chuanyang Jin +3
Effective real-world assistance requires AI agents with robust Theory of Mind (ToM): inferring human mental states from their behavior. Despite recent advances, several key challen…
ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
Chuanyang Jin, Binze Li, Haopeng Xie +6
Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale datas…
The Era of Real-World Human Interaction: RL from User Conversations
Chuanyang Jin, Jing Xu, Bo Liu +6
We posit that to achieve continual model improvement and multifaceted alignment, future models must learn from natural human interaction. Current conversational models are aligned…
ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback
Matteo Bortoletto, Yichao Zhou, Lance Ying +2
While humans are inherently social creatures, the challenge of identifying when and how to assist and collaborate with others - particularly when pursuing independent goals - can h…
Augmented Vision-Language Models: A Systematic Review
Anthony C Davis, Burhan Sadiq, Tianmin Shu +1
Recent advances in visual-language machine learning models have demonstrated exceptional ability to use natural language and understand visual scenes by training on large, unstruct…
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation
Qiyue Gao, Xinyu Pi, Kevin Liu +21
Internal world models (WMs) enable agents to understand the world's state and predict transitions, serving as the basis for advanced deliberative reasoning. Recent large Vision-Lan…
Position: Foundation Models Need Digital Twin Representations
Yiqing Shen, Hao Ding, Lalithkumar Seenivasan +2
Current foundation models (FMs) rely on token representations that directly fragment continuous real-world multimodal data into discrete tokens. They limit FMs to learning real-wor…
RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users
Suyu Ye, Haojun Shi, Darren Shih +3
To achieve successful assistance with long-horizon web-based tasks, AI agents must be able to sequentially follow real-world user instructions over a long period. Unlike existing w…
On Benchmarking Human-Like Intelligence in Machines
Lance Ying, Katherine M. Collins, Lionel Wong +6
Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as…
AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling
Zhining Zhang, Chuanyang Jin, Mung Yao Jia +2
Theory of Mind (ToM), the ability to understand people's minds based on their behavior, is key to developing socially intelligent agents. Current approaches to ToM reasoning either…
Pragmatic Embodied Spoken Instruction Following in Human-Robot Collaboration with Theory of Mind
Lance Ying, Xinyi Li, Shivam Aarya +6
Spoken language instructions are ubiquitous in agent collaboration. However, in real-world human-robot collaboration, following human spoken instructions can be challenging due to…
MuMA-ToM: Multi-modal Multi-Agent Theory of Mind
Haojun Shi, Suyu Ye, Xinyu Fang +4
Understanding people's social interactions in complex real-world scenarios often relies on intricate mental reasoning. To truly understand how and why people interact with one anot…
GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment
Lance Ying, Kunal Jha, Shivam Aarya +3
Verbal communication plays a crucial role in human cooperation, particularly when the partners only have incomplete information about the task, environment, and each other's mental…
MMToM-QA: Multimodal Theory of Mind Question Answering
Chuanyang Jin, Yutong Wu, Jing Cao +7
Theory of Mind (ToM), the ability to understand people's mental states, is an essential ingredient for developing machines with human-level social intelligence. Recent machine lear…
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning
Zhiting Hu, Tianmin Shu
Despite their tremendous success in many applications, large language models often fall short of consistent reasoning and planning in various (language, embodied, and social) scena…