27 citations · 68 across the 24 of their papers we have counts for
26 papers
From Patching to Pruning Visual Computation in Vision Language Models
Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen +3
Vision language models (VLMs) incur substantial inference cost because every visual token is processed by the attention and MLP projections of every decoder layer, even when token-…
ScAle: Attention Head Scaling as a Minimal Adapter for Spatial Reasoning in Vision Language Models
Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen +2
Spatial reasoning remains a persistent challenge for many vision language models (VLMs), and improving it typically requires fine-tuning with substantial additional parameters. Our…
When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making
Jun Liu, Pu Zhao, Zhenglun Kong +12
Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning, planning, and decision-making during interactions with the en…
Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA
Pu Zhao, Arash Akbari, Xuan Shen +16
Recently, Large Language Models (LLMs) have undergone a significant transformation, marked by a rapid rise in both their popularity and capabilities. Leading this evolution are pro…
TSLA: A Task-Specific Learning Adaptation for Semantic Segmentation on Autonomous Vehicles Platform
Jun Liu, Zhenglun Kong, Pu Zhao +9
Autonomous driving platforms encounter diverse driving scenarios, each with varying hardware resources and precision requirements. Given the computational limitations of embedded d…
Enabling Flexible Multi-LLM Integration for Scalable Knowledge Aggregation
Zhenglun Kong, Zheng Zhan, Shiyue Hou +10
Large language models (LLMs) have shown remarkable promise but remain challenging to continually improve through traditional finetuning, particularly when integrating capabilities…
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
Jun Liu, Zhenglun Kong, Peiyan Dong +10
Fine-tuning helps large language models (LLM) recover degraded information and enhance task performance. Although Low-Rank Adaptation (LoRA) is widely used and effective for fine-t…
7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement
Pu Zhao, Xuan Shen, Zhenglun Kong +16
Recently, Large Language Models (LLMs) have undergone a significant transformation, marked by a rapid rise in both their popularity and capabilities. Leading this evolution are pro…
Search for Efficient Large Language Models
Xuan Shen, Pu Zhao, Yifan Gong +7
Large Language Models (LLMs) have long held sway in the realms of artificial intelligence research. Numerous efficient techniques, including weight pruning, quantization, and disti…
Pruning Foundation Models for High Accuracy without Retraining
Pu Zhao, Fei Sun, Xuan Shen +4
Despite the superior performance, it is challenging to deploy foundation models or large language models (LLMs) due to their massive parameters and computations. While pruning is a…
Rethinking Token Reduction for State Space Models
Zheng Zhan, Yushu Wu, Zhenglun Kong +6
Recent advancements in State Space Models (SSMs) have attracted significant interest, particularly in models optimized for parallel training and handling long-range dependencies. A…
Exploring Token Pruning in Vision State Space Models
Zheng Zhan, Zhenglun Kong, Yifan Gong +8
State Space Models (SSMs) have the advantage of keeping linear computational complexity compared to attention modules in transformers, and have been applied to vision tasks as a ne…
AyE-Edge: Automated Deployment Space Search Empowering Accuracy yet Efficient Real-Time Object Detection on the Edge
Chao Wu, Yifan Gong, Liangkai Liu +7
Object detection on the edge (Edge-OD) is in growing demand thanks to its ever-broad application prospects. However, the development of this field is rigorously restricted by the d…
Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
Jun Liu, Zhenglun Kong, Pu Zhao +9
Structured pruning for large language models (LLMs) has garnered significant academic interest due to its ability to efficiently compress and accelerate LLMs by eliminating redunda…
Squat: Quant Small Language Models on the Edge
Xuan Shen, Peiyan Dong, Zhenglun Kong +9
A growing trend has emerged in designing high-quality Small Language Models (SLMs) with a few million parameters. This trend is driven by the increasing concerns over cloud costs,…
Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
Xuan Shen, Peiyan Dong, Lei Lu +5
Large Language Models (LLMs) stand out for their impressive performance in intricate language modeling tasks. However, their demanding computational and memory needs pose obstacles…
DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network
Xuan Shen, Yaohua Wang, Ming Lin +4
The rapid advances in Vision Transformer (ViT) refresh the state-of-the-art performances in various vision tasks, overshadowing the conventional CNN-based models. This ignites a fe…
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Zhenglun Kong, Haoyu Ma, Geng Yuan +12
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit the…
Data Level Lottery Ticket Hypothesis for Vision Transformers
Xuan Shen, Zhenglun Kong, Minghai Qin +6
The conventional lottery ticket hypothesis (LTH) claims that there exists a sparse subnetwork within a dense neural network and a proper random initialization method called the win…
Real-Time Portrait Stylization on the Edge
Yanyu Li, Xuan Shen, Geng Yuan +5
In this work we demonstrate real-time portrait stylization, specifically, translating self-portrait into cartoon or anime style on mobile devices. We propose a latency-driven diffe…
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Zhenglun Kong, Peiyan Dong, Xiaolong Ma +9
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in ind…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
Geng Yuan, Zhiheng Liao, Xiaolong Ma +11
Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vecto…
Towards Fast and Accurate Multi-Person Pose Estimation on Mobile Devices
Xuan Shen, Geng Yuan, Wei Niu +5
The rapid development of autonomous driving, abnormal behavior detection, and behavior recognition makes an increasing demand for multi-person pose estimation-based applications, e…