4 papers
Dissecting GPU Utilization for LLM Inference on Nvidia Hopper
Mohammad Siavashi, Gerald Q. Maguire, Dejan Kostic +1
A single SM utilization percentage can make an LLM inference workload look compute-saturated while hiding how much useful work is being done. The problem is not that the counter is…
Avoiding Cross-Datacenter Collective Congestion via Disaggregated Buffering
Mariano Scazzariello, Noga H. Rotman, Dima Gavrilenko +6
LLM training at the scale of tens of thousands of GPUs now spans multiple datacenters (DC), making cross-DC collectives over long-haul links unavoidable. A critical and overlooked…
Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC
Mohammad Siavashi, Mariano Scazzariello, Gerald Q. Maguire +2
Large Language Model (LLM) inference is rapidly becoming a core datacenter service, yet current serving stacks keep the host CPU on the critical path for orchestration and token-le…
Dissect-and-Restore: AI-based Code Verification with Transient Refactoring
Changjie Wang, Mariano Scazzariello, Anoud Alshnakat +3
Formal verification is increasingly recognized as a critical foundation for building reliable software systems. However, the need for specialized expertise to write precise specifi…