2 citations · 2 across the 10 of their papers we have counts for
11 papers
The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs
Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh +2
Despite strong performance on many multimodal tasks, vision-language models (VLMs) still struggle with basic object counting. We investigate whether this reflects missing internal…
WaveSSM: Multiscale State-Space Models for Non-stationary Signal Attention
Ruben Solozabal, Velibor Bojkovic, Hilal Alquabeh +3
State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be…
Sycophancy Hides Linearly in the Attention Heads
Rifo Genadi, Munachiso Nwadike, Nurdaulet Mukhituly +3
We find that correct-to-incorrect sycophancy signals are most linearly separable within multi-head attention activations. Motivated by the linear representation hypothesis, we trai…
Uncovering the Spectral Bias in Diagonal State Space Models
Ruben Solozabal, Velibor Bojkovic, Hilal AlQuabeh +2
Current methods for initializing state space models (SSMs) parameters mainly rely on the \textit{HiPPO framework}, which is based on an online approximation of orthogonal polynomia…
Emergence of Primacy and Recency Effect in Mamba: A Mechanistic Point of View
Muhammad Cendekia Airlangga, Hilal AlQuabeh, Munachiso S Nwadike +1
We study memory in state-space language models using primacy and recency effects as behavioral tools to uncover how information is retained and forgotten over time. Applying struct…
Mechanistic Insights into Grokking from the Embedding Layer
H. V. AlquBoj, Hilal AlQuabeh, Velibor Bojkovic +2
Grokking, a delayed generalization in neural networks after perfect training performance, has been observed in Transformers and MLPs, but the components driving it remain underexpl…
Number Representations in LLMs: A Computational Parallel to Human Perception
H. V. AlquBoj, Hilal AlQuabeh, Velibor Bojkovic +4
Humans are believed to perceive numbers on a logarithmic mental number line, where smaller values are represented with greater resolution than larger ones. This cognitive bias, sup…
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh +2
This paper investigates whether large language models (LLMs) utilize numerical attributes encoded in a low-dimensional subspace of the embedding space when answering questions invo…
Computational Complexity of Sub-Linear Convergent Algorithms
Hilal AlQuabeh, Farha AlBreiki, Dilshod Azizov
Optimizing machine learning algorithms that are used to solve the objective function has been of great interest. Several approaches to optimize common algorithms, such as gradient…
Pairwise Learning via Stagewise Training in Proximal Setting
Hilal AlQuabeh, Aliakbar Abdurahimov
The pairwise objective paradigms are an important and essential aspect of machine learning. Examples of machine learning approaches that use pairwise objective functions include di…
Investigating a Baseline Of Self Supervised Learning Towards Reducing Labeling Costs For Image Classification
Hilal AlQuabeh, Ameera Bawazeer, Abdulateef Alhashmi
Data labeling in supervised learning is considered an expensive and infeasible tool in some conditions. The self-supervised learning method is proposed to tackle the learning effec…