2 citations · 3 across the 5 of their papers we have counts for
6 papers
Feedback That Backfires in Games: Do LLM Players Repeat the Move They Just Watched Fail?
Esmail Gumaan
When an LLM agent's move fails, the harness writes the move and the failure message into the transcript and asks again. In tool calling, this record makes small models more likely…
Mixture of Attention Schemes (MoAS): Learning to Route Between MHA, GQA, and MQA
Esmail Gumaan
The choice of attention mechanism in Transformer models involves a critical trade-off between modeling quality and inference efficiency. Multi-Head Attention (MHA) offers the best…
Theoretical Foundations and Mitigation of Hallucination in Large Language Models
Esmail Gumaan
Hallucination in Large Language Models (LLMs) refers to the generation of content that is not faithful to the input or the real-world facts. This paper provides a rigorous treatmen…
Universal Approximation Theorem for a Single-Layer Transformer
Esmail Gumaan
Deep learning employs multi-layer neural networks trained via the backpropagation algorithm. This approach has achieved success across many domains and relies on adaptive gradient…
ExpertRAG: Efficient RAG with Mixture of Experts -- Optimizing Context Retrieval for Adaptive LLM Responses
Esmail Gumaan
ExpertRAG is a novel theoretical framework that integrates Mixture-of-Experts (MoE) architectures with Retrieval Augmented Generation (RAG) to advance the efficiency and accuracy o…
Galvatron: Automatic Distributed Training for Large Transformer Models
Esmail Gumaan
Training multi-billion to trillion-parameter language models efficiently on GPU clusters requires leveraging multiple parallelism strategies. We present Galvatron, a novel open-sou…