6 papers
HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models
Haiyan Jiang, Deyu Zhang, Dongdong Weng +2
3D layout generation and editing play a crucial role in Embodied AI and immersive VR interaction. However, manual creation requires tedious labor, while data-driven generation ofte…
Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
Fucheng Jia, Zewen Wu, Shiqi Jiang +7
Large language models (LLMs) are increasingly being deployed on mobile devices, but the limited DRAM capacity constrains the deployable model size. This paper introduces ActiveFlow…
KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse
Huan Yang, Renji Zhang, Mingzhe Huang +5
Recent advances in long-text understanding have pushed the context length of large language models (LLMs) up to one million tokens. It boosts LLMs's accuracy and reasoning capacity…
MobiFuse: A High-Precision On-device Depth Perception System with Multi-Data Fusion
Jinrui Zhang, Deyu Zhang, Tingting Long +6
We present MobiFuse, a high-precision depth perception system on mobile devices that combines dual RGB and Time-of-Flight (ToF) cameras. To achieve this, we leverage physical princ…
A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage
Huan Yang, Deyu Zhang, Yudong Zhao +2
Running LLMs on end devices has garnered significant attention recently due to their advantages in privacy preservation. With the advent of lightweight LLM models and specially des…
Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations
Fucheng Jia, Shiqi Jiang, Ting Cao +9
Web is increasingly becoming the primary platform to deliver AI services onto edge devices, making in-browser deep learning (DL) inference more prominent. Nevertheless, the heterog…