4 papers
Context-aware Rotary Position Embedding
Ali Veisi, Delaram Fartoot, Hamidreza Amirzadeh
Positional encoding is a vital component of Transformer architectures, enabling models to incorporate sequence order into self-attention mechanisms. Rotary Positional Embeddings (R…
How Language Models Prioritize Contextual Grammatical Cues?
Hamidreza Amirzadeh, Afra Alishahi, Hosein Mohebbi
Transformer-based language models have shown an excellent ability to effectively capture and utilize contextual information. Although various analysis techniques have been used to…
data2lang2vec: Data Driven Typological Features Completion
Hamidreza Amirzadeh, Sadegh Jafari, Anika Harju +1
Language typology databases enhance multi-lingual Natural Language Processing (NLP) by improving model adaptability to diverse linguistic structures. The widely-used lang2vec toolk…
Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems
Zeinab Sadat Taghavi, Soroush Gooran, Seyed Arshan Dalili +3
In this paper, we introduce a novel Artificial Intelligence (AI) system inspired by the philosophical and psychoanalytical concept of imagination as a ``Re-construction of Experien…