42 citations · 58 across the 6 of their papers we have counts for
6 papers
Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
Haojun Xia, Xiaoxia Wu, Jisen Li +12
The KV cache is a dominant memory bottleneck for LLM inference. While 4-bit KV quantization preserves accuracy, 2-bit often degrades it, especially on long-context reasoning. We cl…
Alopex: A Computational Framework for Enabling On-Device Function Calls with LLMs
Yide Ran, Zhaozhuo Xu, Yuhang Yao +9
The rapid advancement of Large Language Models (LLMs) has led to their increased integration into mobile devices for personalized assistance, which enables LLMs to call external AP…
ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
Yuhang Yao, Han Jin, Alay Dilipbhai Shah +7
Large language models (LLMs) have surged in popularity and are extensively used in commercial applications, where the efficiency of model serving is crucial for the user experience…
TensorOpera Router: A Multi-Model Router for Efficient LLM Inference
Dimitris Stripelis, Zijian Hu, Jipeng Zhang +6
With the rapid growth of Large Language Models (LLMs) across various domains, numerous new LLMs have emerged, each possessing domain-specific expertise. This proliferation has high…
FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
Chaoyang He, Alay Dilipbhai Shah, Zhenheng Tang +7
Federated Learning (FL) is a distributed learning paradigm that can learn a global or personalized model from decentralized datasets on edge devices. However, in the computer visio…
Predicting Future Sales of Retail Products using Machine Learning
Devendra Swami, Alay Dilipbhai Shah, Subhrajeet K B Ray
Techniques for making future predictions based upon the present and past data, has always been an area with direct application to various real life problems. We are discussing a si…