30 citations · 31 across the 8 of their papers we have counts for
5 papers · 1 filter
SparseAccelerate: Efficient Long-Context Inference for Mid-Range GPUs
James Vo
As Large Language Models (LLMs) scale to longer context windows, the computational cost of attention mechanisms, which traditionally grows quadratically with input length, presents…
Combining Entropy and Matrix Nuclear Norm for Enhanced Evaluation of Language Models
James Vo
As large language models (LLMs) continue to advance, the need for precise and efficient evaluation metrics becomes more pressing. Traditional approaches, while informative, often f…
Transformer Layer Injection: A Novel Approach for Efficient Upscaling of Large Language Models
James Vo
In this paper, we propose Transformer Layer Injection (TLI), a novel method for efficiently upscaling large language models (LLMs) while minimizing computational costs and maintain…
Efficient Second-Order Neural Network Optimization via Adaptive Trust Region Methods
James Vo
Second-order optimization methods offer notable advantages in training deep neural networks by utilizing curvature information to achieve faster convergence. However, traditional s…
Vi-Mistral-X: Building a Vietnamese Language Model with Advanced Continual Pre-training
James Vo
The advancement of Large Language Models (LLMs) has significantly transformed the field of natural language processing, although the focus on English-centric models has created a n…