10 papers
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
WenHung Lee, Jian-Jia Chen, Xiaolin Lin +6
While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck whe…
Jump Like A Squirrel: Optimized Execution Step Order for Anytime Random Forest Inference
Daniel Biebert, Christian Hakert, Kay Heider +3
Due to their efficiency and small size, decision trees and random forests are popular machine learning models used for classification on resource-constrained systems. In such syste…
Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
Pei-Shuo Wang, Jian-Jia Chen, Chun-Che Yang +4
The immense model sizes of large language models (LLMs) challenge deployment on memory-limited consumer GPUs. Although model compression and parameter offloading are common strateg…
Modeling and Simulating Emerging Memory Technologies: A Tutorial
Yun-Chih Chen, Tristan Seidl, Nils Hölscher +15
Non-volatile Memory (NVM) technologies present a promising alternative to traditional volatile memories such as SRAM and DRAM. Due to the limited availability of real NVM devices,…
CHRONOS: Compensating Hardware Related Overheads with Native Multi Timer Support for Real-Time Operating Systems
Kay Heider, Christian Hakert, Kuan-Hsun Chen +1
The management of timing constraints in a real-time operating system (RTOS) is usually realized through a global tick counter. This counter acts as the foundational time unit for a…
: A Racetrack Register File with Runtime Software Reconfiguration
Christian Hakert, Shuo-Han Chen, Kay Heider +4
Arising disruptive memory technologies continuously make their way into the memory hierarchy at various levels. Racetrack memory is one promising candidate for future memory due to…
Realizing Hardware-Optimized General Tree-Based Data Structures for Heterogeneous System Classes
Daniel Biebert, Christian Hakert, Jian-Jia Chen
Tree-based data structures are ubiquitous across applications. Therefore, a multitude of different tree implementations exist. However, while these implementations are diverse, the…
WCDT: Systematic WCET Optimization for Decision Tree Implementations
Nils Hölscher, Christian Hakert, Georg von der Brüggen +3
Machine-learning models are increasingly deployed on resource-constrained embedded systems with strict timing constraints. In such scenarios, the worst-case execution time (WCET) o…
On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning
Simon Kurz, Jian-Jia Chen, Lucie Flek +1
Recent advances in large language model (LLM) pruning have shown state-of-the-art (SotA) compression results in post-training and retraining-free settings while maintaining high pr…
Register Your Forests: Decision Tree Ensemble Optimization by Explicit CPU Register Allocation
Daniel Biebert, Christian Hakert, Kuan-Hsun Chen +1
Bringing high-level machine learning models to efficient and well-suited machine implementations often invokes a bunch of tools, e.g.~code generators, compilers, and optimizers. Al…