8 papers · 1 filter
Low-Precision Rank Compensation for Matrices and Tensor Trains
Ibrahim El Mountasser, Khalide Jbilou
Lower numerical precision reduces storage and memory traffic but raises the perturbation floor. We study rank compensation: reinvesting saved memory in a larger approximation rank.…
Structured Multidimensional Representation Learning for Large Language Models
Alaa El Ichi, Khalide Jbilou, Mohamed El Guide +1
Transformer architectures achieve state-of-the-art performance across a wide range of pattern recognition and natural language processing tasks, but their scaling is accompanied by…
A Computationally Efficient Multidimensional Vision Transformer
Alaa El Ichi, Khalide Jbilou
Vision Transformers have achieved state-of-the-art performance in a wide range of computer vision tasks, but their practical deployment is limited by high computational and memory…
Dimensionality Reduction on Riemannian Manifolds in Data Analysis
Alaa El Ichi, Khalide Jbilou
In this work, we investigate Riemannian geometry based dimensionality reduction methods that respect the underlying manifold structure of the data. In particular, we focus on Princ…
Multidimensional Task Learning: A Unified Tensor Framework for Computer Vision Tasks
Alaa El Ichi, Khalide Jbilou
This paper introduces Multidimensional Task Learning (MTL), a unified mathematical framework based on Generalized Einstein MLPs (GE-MLPs) that operate directly on tensors via the E…
A survey of scalar and vector extrapolation
Khalide Jbilou
Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly c…
Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization
A. El Ichi, K. Jbilou
Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language tasks but suffer from extremely large memory footprints and computational cos…
A Unified Trace-Optimization Framework for Multidimensionality Reduction
Mohamed El Guide, Alaa El Ichi, Khalide Jbilou +2
This paper presents a comprehensive overview of several multidimensional reduction methods focusing on Multidimensional Principal Component Analysis (MPCA), Multilinear Orthogonal…