6 papers
Deterministic LLM Inference Across GPU Kernels: Power-of-Two INT8 Quantization Scales and the Limits of Tolerance-Based Conformance
Teng-Ruei Chen
Conformance suites for quantized GEMM kernels ask whether two implementations agree within a tolerance. We measure what such a suite can detect. Injecting nine faults into a refere…
The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference
Teng-Ruei Chen
Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable. We test that assumption: holding the checkpoint, prompts, hardware, inferen…
Spec Sheets Are Not Kernels: An ISA- and Source-Level Audit of INT8 Availability on NVIDIA Blackwell Ultra
Teng-Ruei Chen
NVIDIA's published specifications give the Blackwell Ultra GPU (B300) a dense-compute ratio of roughly 30:1 between FP8 and INT8 tensor-core throughput; its predecessors, H200 and…
What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings
Teng-Ruei Chen
Routing benchmarks often compare a policy that commits to one model before seeing its response with a hindsight oracle that credits any correct recorded output. Under stochastic de…
Resample or Reroute? Recoverable Stopping Debt Without Identified Action Selection
Teng-Ruei Chen
After a weak verifier accepts a large-language-model response, a second call may resample or reroute. Because correctness is hidden, action selection is an identification problem.…
Test-Time Alignment for Large Language Models via Textual Model Predictive Control
Kuang-Da Wang, Teng-Ruei Chen, Yu Heng Hung +7
Aligning Large Language Models (LLMs) with human preferences through finetuning is resource-intensive, motivating lightweight alternatives at test time. We address test-time alignm…