9 papers
Avoiding Cross-Datacenter Collective Congestion via Disaggregated Buffering
Mariano Scazzariello, Noga H. Rotman, Dima Gavrilenko +6
LLM training at the scale of tens of thousands of GPUs now spans multiple datacenters (DC), making cross-DC collectives over long-haul links unavoidable. A critical and overlooked…
Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC
Mohammad Siavashi, Mariano Scazzariello, Gerald Q. Maguire +2
Large Language Model (LLM) inference is rapidly becoming a core datacenter service, yet current serving stacks keep the host CPU on the critical path for orchestration and token-le…
KVComm: Enabling Efficient LLM Communication through Selective KV Sharing
Xiangyu Shi, Marco Chiesa, Gerald Q. Maguire +1
Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial. Existing communication protocols either rely on…
Dissect-and-Restore: AI-based Code Verification with Transient Refactoring
Changjie Wang, Mariano Scazzariello, Anoud Alshnakat +3
Formal verification is increasingly recognized as a critical foundation for building reliable software systems. However, the need for specialized expertise to write precise specifi…
Can LLMs Forecast Internet Traffic from Social Media?
Jonatan Langlet, Mariano Scazzariello, Flavio Luciani +3
Societal events shape the Internet's behavior. The death of a prominent public figure, a software launch, or a major sports match can trigger sudden demand surges that overwhelm pe…
Priority-Aware Preemptive Scheduling for Mixed-Priority Workloads in MoE Inference
Mohammad Siavashi, Faezeh Keshmiri Dindarloo, Dejan Kostic +1
Large Language Models have revolutionized natural language processing, yet serving them efficiently in data centers remains challenging due to mixed workloads comprising latency-se…