6 papers
VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
Zihao Zheng, Zhihao Mao, Xingyue Zhou +9
Vision-and-Language Navigation (VLN) increasingly relies on large vision-language models, but their inference cost conflicts with real-time deployment. Token caching is a promising…
HeiSD: Hybrid Speculative Decoding for Embodied Vision-Language-Action Models with Kinematic Awareness
Zihao Zheng, Zhihao Mao, Sicheng Tian +8
Vision-Language-Action (VLA) Models have become the mainstream solution for robot control, but suffer from slow inference speeds. Speculative Decoding (SD) is a promising accelerat…
KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Models
Zihao Zheng, Zhihao Mao, Maoliang Li +6
Vision-Language-Action (VLA) models build a token-domain robot control paradigm, yet suffer from low speed. Speculative Decoding (SD) is an optimization strategy that can boost inf…
DyQ-VLA: Temporal-Dynamic-Aware Quantization for Embodied Vision-Language-Action Models
Zihao Zheng, Hangyu Cao, Sicheng Tian +9
Vision-Language-Action (VLA) models are dominant in embodied intelligence but are constrained by inference overheads. While model quantization alleviates these bottlenecks for edge…
SpotTune: Leveraging Transient Resources for Cost-efficient Hyper-parameter Tuning in the Public Cloud
Yan Li, Bo An, Junming Ma +3
Hyper-parameter tuning (HPT) is crucial for many machine learning (ML) algorithms. But due to the large searching space, HPT is usually time-consuming and resource-intensive. Nowad…
S3ML: A Secure Serving System for Machine Learning Inference
Junming Ma, Chaofan Yu, Aihui Zhou +6
We present S3ML, a secure serving system for machine learning inference in this paper. S3ML runs machine learning models in Intel SGX enclaves to protect users' privacy. S3ML desig…