collaborators

1 paper

cs.CL2024

Kernel Looping: Eliminating Synchronization Boundaries for Peak Inference Performance

David Koeplinger, Darshan Gandhi, Pushkar Nandkar +9

Token generation speed is critical to power the next wave of AI inference applications. GPUs significantly underperform during token generation due to synchronization overheads at…