4 papers · 1 filter
Online Learning for Multi-Layer Hierarchical Inference under Partial and Policy-Dependent Feedback
Haoran Zhang, Seohyeon Cha, Hasan Burhan Beytur +3
Hierarchical inference systems route tasks across multiple computational layers, where each node may either finalize a prediction locally or offload the task to a node in the next…
CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization
Seohyeon Cha, Huancheng Chen, Dongjun Kim +4
Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats without retraining, typically using a small…
Optimal Resource Allocation for ML Model Training and Deployment under Concept Drift
Hasan Burhan Beytur, Haris Vikalo, Kevin S Chan +1
We study how to allocate resources for training and deployment of machine learning (ML) models under concept drift and limited budgets. We consider a setting in which a model provi…
Batching-Aware Joint Model Onloading and Offloading for Hierarchical Multi-Task Inference
Seohyeon Cha, Kevin Chan, Gustavo de Veciana +1
The growing demand for intelligent services on resource-constrained edge devices has spurred the development of collaborative inference systems that distribute workloads across end…