9 papers
MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration
Kosmas Alexandridis, Giorgos Dimitrakopoulos
The rapid growth of Large Language Models (LLMs) has intensified the need for specialized hardware accelerators that can satisfy stringent inference latency and power constraints.…
H-FA: A Hybrid Floating-Point and Logarithmic Approach to Hardware Accelerated FlashAttention
Kosmas Alexandridis, Giorgos Dimitrakopoulos
Transformers have significantly advanced AI and machine learning through their powerful attention mechanism. However, computing attention on long sequences can become a computation…
Vectorized FlashAttention with Low-cost Exponential Computation in RISC-V Vector Processors
Vasileios Titopoulos, Kosmas Alexandridis, Giorgos Dimitrakopoulos
Attention is a core operation in numerous machine learning and artificial intelligence models. This work focuses on the acceleration of attention kernel using FlashAttention algori…
Custom Algorithm-based Fault Tolerance for Attention Layers in Transformers
Vasileios Titopoulos, Kosmas Alexandridis, Giorgos Dimitrakopoulos
Transformers and large language models (LLMs), powered by the attention mechanism, have transformed numerous AI applications, driving the need for specialized hardware accelerators…
Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
Kosmas Alexandridis, Vasileios Titopoulos, Giorgos Dimitrakopoulos
Attention mechanisms, particularly within Transformer architectures and large language models (LLMs), have revolutionized sequence modeling in machine learning and artificial intel…
FLASH-D: FlashAttention with Hidden Softmax Division
Kosmas Alexandridis, Vasileios Titopoulos, Giorgos Dimitrakopoulos
The transformer's attention mechanism has revolutionized AI and machine learning, with its efficient computation being crucial to its performance. However, calculating attention in…