FlexQuant: Elastic Quantization Framework for Locally Hosted LLM on Edge Devices

Yuji Chai, Mujin Kwen, Mujin Kwun +2

Deploying LLMs on edge devices presents serious technical challenges. Memory elasticity is crucial for edge devices with unified memory, where memory is shared and fluctuates dynam…