204 citations · 509 across the 65 of their papers we have counts for
42 papers · 1 filter
AgentKV: Phase-Aware KV Eviction for Agentic LLMs
Taowen Tony Liu, Jeffrey T. H. Wong, Can Xiao +3
Agentic serving can consume orders of magnitude more tokens than chatbot workloads, stressing both KV-cache capacity and decode-time bandwidth. Most KV-eviction methods score cache…
DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs
Zeyu Cao, Xuan Guo, Cheng Zhang +3
As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a produ…
TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
Hanzhang Shen, Haoran Wu, Yiren Zhao +1
Agentic workloads have emerged as a major workload for LLM inference. They differ significantly from chat-only workloads, requiring long-context processing, the ability to handle m…
Deep Kernel Fusion for Transformers
Zixi Zhang, Zhiwen Mo, Yiren Zhao +1
Agentic LLM inference with long contexts is increasingly limited by memory bandwidth rather than compute. In this setting, SwiGLU MLP blocks, whose large weights exceed cache capac…
On the Existence and Behavior of Secondary Attention Sinks
Jeffrey T. H. Wong, Cheng Zhang, Louis Mahon +3
Attention sinks are tokens, often the beginning-of-sequence (BOS) token, that receive disproportionately high attention despite limited semantic relevance. In this work, we identif…
Hardware and Software Platform Inference
Cheng Zhang, Hanna Foerster, Robert D. Mullins +2
It is now a common business practice to buy access to large language model (LLM) inference rather than self-host, because of significant upfront hardware infrastructure and energy…
QERA: an Analytical Framework for Quantization Error Reconstruction
Cheng Zhang, Jeffrey T. H. Wong, Can Xiao +2
The growing number of parameters and computational demands of large language models (LLMs) present significant challenges for their efficient deployment. Recently, there is an incr…
GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning
Zehui Li, Vallijah Subasri, Guy-Bart Stan +2
Genetic variants (GVs) are defined as differences in the DNA sequences among individuals and play a crucial role in diagnosing and treating genetic diseases. The rapid decrease in…
Optimised Grouped-Query Attention Mechanism for Transformers
Yuang Chen, Cheng Zhang, Xitong Gao +3
Grouped-query attention (GQA) has been widely adopted in LLMs to mitigate the complexity of multi-head attention (MHA). To transform an MHA to a GQA, neighbour queries in MHA are e…
Unlocking the Global Synergies in Low-Rank Adapters
Zixi Zhang, Cheng Zhang, Xitong Gao +3
Low-rank Adaption (LoRA) has been the de-facto parameter-efficient fine-tuning technique for large language models. We present HeteroLoRA, a light-weight search algorithm that leve…
Enhancing Node Representations for Real-World Complex Networks with Topological Augmentation
Xiangyu Zhao, Zehui Li, Mingzhu Shen +3
Graph augmentation methods play a crucial role in improving the performance and enhancing generalisation capabilities in Graph Neural Networks (GNNs). Existing graph augmentation m…
LQER: Low-Rank Quantization Error Reconstruction for LLMs
Cheng Zhang, Jianyi Cheng, George A. Constantinides +1
Post-training quantization of Large Language Models (LLMs) is challenging. In this work, we introduce Low-rank Quantization Error Reduction (LQER), which combines quantization and…
MiliPoint: A Point Cloud Dataset for mmWave Radar
Han Cui, Shu Zhong, Jiacheng Wu +3
Millimetre-wave (mmWave) radar has emerged as an attractive and cost-effective alternative for human activity sensing compared to traditional camera-based systems. mmWave radars ar…
Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?
Cheng Zhang, Jianyi Cheng, Ilia Shumailov +2
The inference of Large language models (LLMs) requires immense computation and memory resources. To curtail these costs, quantisation has merged as a promising solution, but existi…
Latent Diffusion Model for DNA Sequence Generation
Zehui Li, Yuhao Ni, Tim August B. Huygelen +4
The harnessing of machine learning, especially deep generative models, has opened up promising avenues in the field of synthetic DNA sequence generation. Whilst Generative Adversar…
LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation
Zixi Zhang, Balint Szekely, Pedro Gimenes +5
Hardware design verification (DV) is a process that checks the functional equivalence of a hardware design against its specifications, improving hardware reliability and robustness…
Will More Expressive Graph Neural Networks do Better on Generative Tasks?
Xiandong Zou, Xiangyu Zhao, Pietro Liò +1
Graph generation poses a significant challenge as it involves predicting a complete graph with multiple nodes and edges based on simply a given label. This task also carries fundam…
Genomic Interpreter: A Hierarchical Genomic Deep Neural Network with 1D Shifted Window Transformer
Zehui Li, Akashaditya Das, William A V Beardall +2
Given the increasing volume and quality of genomics data, extracting new insights requires interpretable machine-learning models. This work presents Genomic Interpreter: a novel ar…
Hybrid Graph: A Unified Graph Representation with Datasets and Benchmarks for Complex Graphs
Zehui Li, Xiangyu Zhao, Mingzhu Shen +3
Graphs are widely used to encapsulate a variety of data formats, but real-world networks often involve complex node relations beyond only being pairwise. While hypergraphs and hier…
The Curse of Recursion: Training on Generated Data Makes Models Forget
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao +3
Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT in…
Dynamic Stashing Quantization for Efficient Transformer Training
Guo Yang, Daniel Lo, Robert Mullins +1
Large Language Models (LLMs) have demonstrated impressive performance on a range of Natural Language Processing (NLP) tasks. Unfortunately, the immense amount of computations and m…
Task-Agnostic Graph Neural Network Evaluation via Adversarial Collaboration
Xiangyu Zhao, Hannes Stärk, Dominique Beaini +2
It has been increasingly demanding to develop reliable methods to evaluate the progress of Graph Neural Network (GNN) research for molecular representation learning. Existing GNN b…
Wide Attention Is The Way Forward For Transformers?
Jason Ross Brown, Yiren Zhao, Ilia Shumailov +1
The Transformer is an extremely powerful and prominent deep learning architecture. In this work, we challenge the commonly held belief in deep learning that going deeper is better,…
Revisiting Structured Dropout
Yiren Zhao, Oluwatomisin Dada, Xitong Gao +1
Large neural networks are often overparameterised and prone to overfitting, Dropout is a widely used regularization technique to combat overfitting and improve model generalization…