37 citations · 171 across the 17 of their papers we have counts for
17 papers
MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs
Zhenyan Lu, Daliang Xu, Dongqi Cai +5
Large language models (LLMs) are deployed on mobile devices to power killer applications such as intelligent assistants. LLMs pre-trained on general corpora often hallucinate when…
Evidencing Unauthorized Training Data from AI Generated Content using Information Isotopes
Qi Tao, Yin Jinhua, Cai Dongqi +10
In light of scaling laws, many AI institutions are intensifying efforts to construct advanced AIs on extensive collections of high-quality human data. However, in a rush to stay co…
Recall: Empowering Multimodal Embedding for Edge Devices
Dongqi Cai, Shangguang Wang, Chen Peng +2
Human memory is inherently prone to forgetting. To address this, multimodal embedding models have been introduced, which transform diverse real-world data into a unified embedding…
Small Language Models: Survey, Measurements, and Insights
Zhenyan Lu, Xiang Li, Dongqi Cai +5
Small language models (SLMs), despite their widespread adoption in modern smart devices, have received significantly less academic attention compared to their large language model…
FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts
Hanzi Mei, Dongqi Cai, Ao Zhou +2
As Large Language Models (LLMs) push the boundaries of AI capabilities, their demand for data is growing. Much of this data is private and distributed across edge devices, making F…
ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents
Haiyang Shen, Yue Li, Desong Meng +5
Recent advancements in integrating large language models (LLMs) with application programming interfaces (APIs) have gained significant interest in both academia and industry. Recen…
FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmission
Zeling Zhang, Dongqi Cai, Yiran Zhang +3
Communication overhead is a significant bottleneck in federated learning (FL), which has been exaggerated with the increasing size of AI models. In this paper, we propose FedRDMA,…
Lightweight Protection for Privacy in Offloaded Speech Understanding
Dongqi Cai
Speech is a common input method for mobile embedded devices, but cloud-based speech recognition systems pose privacy risks. Disentanglement-based encoders, designed to safeguard us…
A Survey of Resource-efficient LLM and Multimodal Foundation Models
Mengwei Xu, Wangsong Yin, Dongqi Cai +15
Large foundation models, including large language models (LLMs), vision transformers (ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine lear…
Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition
Dongqi Cai, Yangyuxuan Kang, Anbang Yao +1
This paper presents Ske2Grid, a new representation learning framework for improved skeleton-based action recognition. In Ske2Grid, we define a regular convolution operation upon a…
Mobile Foundation Model as Firmware
Jinliang Yuan, Chen Yang, Dongqi Cai +9
In today's landscape, smartphones have evolved into hubs for hosting a multitude of deep learning models aimed at local execution. A key realization driving this work is the notabl…
FwdLLM: Efficient FedLLM using Forward Gradient
Mengwei Xu, Dongqi Cai, Yaozong Wu +2
Large Language Models (LLMs) are transforming the landscape of mobile intelligence. Federated Learning (FL), a method to preserve user data privacy, is often employed in fine-tunin…
Federated Few-Shot Learning for Mobile NLP
Dongqi Cai, Shangguang Wang, Yaozong Wu +2
Natural language processing (NLP) sees rich mobile applications. To support various language understanding tasks, a foundation NLP model is often fine-tuned in a federated, privacy…
Towards Practical Few-shot Federated NLP
Dongqi Cai, Yaozong Wu, Haitao Yuan +3
Transformer-based pre-trained models have emerged as the predominant solution for natural language processing (NLP). Fine-tuning such pre-trained models for downstream tasks often…
Accelerating Vertical Federated Learning
Dongqi Cai, Tao Fan, Yan Kang +4
Privacy, security and data governance constraints rule out a brute force process in the integration of cross-silo data, which inherits the development of the Internet of Things. Fe…
FedAdapter: Efficient Federated Learning for Modern NLP
Dongqi Cai, Yaozong Wu, Shangguang Wang +2
Transformer-based pre-trained models have revolutionized NLP for superior performance and generality. Fine-tuning pre-trained models for downstream tasks often requires private dat…
Learning Visual Knowledge Memory Networks for Visual Question Answering
Zhou Su, Chen Zhu, Yinpeng Dong +3
Visual question answering (VQA) requires joint comprehension of images and natural language questions, where many questions can't be directly or clearly answered from visual conten…