94 citations · 200 across the 17 of their papers we have counts for
18 papers
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows
Qiushi Sun, Mukai Li, Zhoumianze Liu +11
Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms. While these agents…
EconProver: Towards More Economical Test-Time Scaling for Automated Theorem Proving
Mukai Li, Linfeng Song, Zhenwen Liang +5
Large Language Models (LLMs) have recently advanced the field of Automated Theorem Proving (ATP), attaining substantial performance gains through widely adopted test-time scaling s…
GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models
Mukai Li, Lei Li, Shansan Gong +1
Visual Language Models (VLMs) demonstrate impressive capabilities in processing multimodal inputs, yet applications such as visual agents, which require handling multiple images an…
VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment
Lei Li, Zhihui Xie, Mukai Li +7
As large vision-language models (LVLMs) evolve rapidly, the demand for high-quality and diverse data to align these models becomes increasingly crucial. However, the creation of su…
Scaling Diffusion Language Models via Adaptation from Autoregressive Models
Shansan Gong, Shivam Agarwal, Yizhe Zhang +9
Diffusion Language Models (DLMs) have emerged as a promising new paradigm for text generative modeling, potentially addressing limitations of autoregressive (AR) models. However, c…
Red Teaming Visual Language Models
Mukai Li, Lei Li, Yuwei Yin +3
VLMs (Vision-Language Models) extend the capabilities of LLMs (Large Language Models) to accept multimodal inputs. Since it has been verified that LLMs can be induced to generate h…
Silkie: Preference Distillation for Large Visual Language Models
Lei Li, Zhihui Xie, Mukai Li +6
This paper explores preference distillation for large vision language models (LVLMs), improving their ability to generate helpful and faithful responses anchoring the visual contex…
DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models
Shansan Gong, Mukai Li, Jiangtao Feng +2
Diffusion models have gained prominence in generating high-quality sequences of text. Nevertheless, current approaches predominantly represent discrete text within a continuous dif…
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Chenxin An, Shansan Gong, Ming Zhong +5
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations wi…
MIT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
Lei Li, Yuwei Yin, Shicheng Li +9
Instruction tuning has significantly advanced large language models (LLMs) such as ChatGPT, enabling them to align with human instructions across diverse tasks. However, progress i…
LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark
Zhenfei Yin, Jiong Wang, Jianjian Cao +9
Large language models have emerged as a promising approach towards achieving general-purpose AI agents. The thriving open-source LLM community has greatly accelerated the developme…
In-Context Learning with Many Demonstration Examples
Mukai Li, Shansan Gong, Jiangtao Feng +4
Large pre-training language models (PLMs) have shown promising in-context learning abilities. However, due to the backbone transformer architecture, existing PLMs are bottlenecked…
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Shansan Gong, Mukai Li, Jiangtao Feng +2
Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusio…
ERGO: Event Relational Graph Transformer for Document-level Event Causality Identification
Meiqi Chen, Yixin Cao, Kunquan Deng +4
Document-level Event Causality Identification (DECI) aims to identify causal relations between event pairs in a document. It poses a great challenge of across-sentence reasoning wi…
Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction
Yubo Ma, Zehao Wang, Yixin Cao +4
In this paper, we propose an effective yet efficient model PAIE for both sentence-level and document-level Event Argument Extraction (EAE), which also generalizes well when there i…
Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer
Fanchao Qi, Yangyi Chen, Xurui Zhang +3
Adversarial attacks and backdoor attacks are two common security threats that hang over deep learning. Both of them harness task-irrelevant features of data in their implementation…
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Fanchao Qi, Mukai Li, Yangyi Chen +4
Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-…
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
Fanchao Qi, Yangyi Chen, Mukai Li +3
Backdoor attacks are a kind of emergent training-time threat to deep neural networks (DNNs). They can manipulate the output of DNNs and possess high insidiousness. In the field of…