3 citations · 6 across the 6 of their papers we have counts for
5 papers · 1 filter
Student-Guided Teacher Distillation for Efficient LLM Task Routing: Positioning Against Jev-Style System-1 Classifiers
Haifeng Wu, Srinivasan Manoharan, Jian Wan +3
Zero-shot classifiers are useful for routing user requests to specialized LLM tasks, but scoring every request against a large candidate set is expensive: a zero-shot NLI classifie…
Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing
Srinivasan Manoharan, Junhua Zhao, Fangbo Tu +6
Enterprise AI coding assistants incur substantial inference spend, and naive token-cost minimization often fails to reduce end-to-end cost once retries, escalations, and developer…
RLM-Cascade: Response-Level Speculative Decoding for Cost-Efficient LLM API Serving
Haifeng Wu, Srinivasan Manoharan, Fangbo Tu +2
We present RLM-Cascade, a proxy-layer system that applies speculative decoding at the response level to reduce LLM API costs without requiring model architecture access or a shared…
Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data
Srinivasan Manoharan, Dilipkumar Nallusamy, Sachin Kumar +1
Deploying frontier large language models (LLMs) for domain-specific structured evaluation tasks incurs prohibitive latency, cost, and data-privacy overhead. We present a hybrid fra…
Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation
Fangbo Tu, Junhua Zhao, Chi Liu +4
Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production enviro…