7 citations · 22 across the 15 of their papers we have counts for
15 papers
3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code
Yipeng Gao, Lei Shu, Genzhi Ye +5
Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neural 3D generators inherently la…
Ego2Web: A Web Agent Benchmark Grounded in Egocentric Videos
Shoubin Yu, Lei Shu, Antoine Yang +6
Multimodal AI agents are increasingly automating complex real-world workflows that involve online web execution. However, current web-agent benchmarks suffer from a critical limita…
Optimizing Generative Ranking Relevance via Reinforcement Learning in Xiaohongshu Search
Ziyang Zeng, Heming Jing, Jindong Chen +11
Ranking relevance is a fundamental task in search engines, aiming to identify the items most relevant to a given user query. Traditional relevance models typically produce scalar s…
UISim: An Interactive Image-Based UI Simulator for Dynamic Mobile Environments
Jiannan Xiang, Yun Zhu, Lei Shu +6
Developing and testing user interfaces (UIs) and training AI agents to interact with them are challenging due to the dynamic and diverse nature of real-world mobile environments. E…
WebQuest: A Benchmark for Multimodal QA on Web Page Sequences
Maria Wang, Srinivas Sunkara, Gilles Baechler +5
The rise of powerful multimodal LLMs has enhanced the viability of building web agents which can, with increasing levels of autonomy, assist users to retrieve information and compl…
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection
Yun Zhu, Jia-Chen Gu, Caitlin Sikora +8
Large language models (LLMs) augmented with retrieval exhibit robust performance and extensive versatility by incorporating external contexts. However, the input length grows linea…
ScreenAI: A Vision-Language Model for UI and Infographics Understanding
Gilles Baechler, Srinivas Sunkara, Maria Wang +7
Screen user interfaces (UIs) and infographics, sharing similar visual language and design principles, play important roles in human communication and human-machine interaction. We…
SiRA: Sparse Mixture of Low Rank Adaptation
Yun Zhu, Nevan Wichers, Chu-Cheng Lin +8
Parameter Efficient Tuning has been an prominent approach to adapt the Large Language Model to downstream tasks. Most previous works considers adding the dense trainable parameters…
Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
Bowen Tan, Yun Zhu, Lijuan Liu +3
Large language models (LLMs) such as T0, FLAN, and OPT-IML, excel in multi-tasking under a unified instruction-following paradigm, where they also exhibit remarkable generalization…
Fusion-Eval: Integrating Assistant Evaluators with LLMs
Lei Shu, Nevan Wichers, Liangchen Luo +4
Evaluating natural language systems poses significant challenges, particularly in the realms of natural language understanding and high-level reasoning. In this paper, we introduce…
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs
Bowen Tan, Yun Zhu, Lijuan Liu +5
The recent progress of AI can be largely attributed to large language models (LLMs). However, their escalating memory requirements introduce challenges for machine learning (ML) re…
DETR++: Taming Your Multi-Scale Detection Transformer
Chi Zhang, Lijuan Liu, Xiaoxue Zang +4
Convolutional Neural Networks (CNN) have dominated the field of detection ever since the success of AlexNet in ImageNet classification [12]. With the sweeping reform of Transformer…
UIBert: Learning Generic Multimodal Representations for UI Understanding
Chongyang Bai, Xiaoxue Zang, Ying Xu +4
To improve the accessibility of smart devices and to simplify their usage, building models which understand user interfaces (UIs) and assist users to complete their tasks is critic…
Multimodal Icon Annotation For Mobile Applications
Xiaoxue Zang, Ying Xu, Jindong Chen
Annotating user interfaces (UIs) that involves localization and classification of meaningful UI elements on a screen is a critical step for many mobile applications such as screen…
PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling
Xiaoxue Zang, Lijuan Liu, Maria Wang +3
We present a new human-human dialogue dataset - PhotoChat, the first dataset that casts light on the photo sharing behavior in onlin emessaging. PhotoChat contains 12k dialogues, e…