3 citations · 5 across the 20 of their papers we have counts for
9 papers · 1 filter
Temporal-Spatial Tubelet Embedding for Cloud-Robust MSI Reconstruction using MSI-SAR Fusion: A Multi-Head Self-Attention Video Vision Transformer Approach
Yiqun Wang, Lujun Li, Meiru Yue +1
Cloud cover in multispectral imagery (MSI) significantly hinders early-season crop mapping by corrupting spectral information. Existing Vision Transformer(ViT)-based time-series re…
NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks
Lama Sleem, Jerome Francois, Lujun Li +3
Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelin…
Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish
Lujun Li, Yewei Song, Lama Sleem +7
Grammar refers to the system of rules that governs the structural organization and the semantic relations among linguistic units such as sentences, phrases, and words within a give…
Uncovering Zero-Shot Generalization Gaps in Time-Series Foundation Models Using Real-World Videos
Lujun Li, Lama Sleem, Yiqun Wang +3
Recent research on time-series foundation models (TSFMs) has underscored the scarcity of real-world data, often supplemented with synthetic sources in existing datasets, whose gene…
Limitations of Normalization in Attention Mechanism
Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1
This paper investigates the limitations of the normalization in attention mechanisms. We begin with a theoretical framework that enables the identification of the model's selective…
Small Language Models in the Real World: Insights from Industrial Text Classification
Lujun Li, Lama Sleem, Niccolo' Gentile +2
With the emergence of ChatGPT, Transformer models have significantly advanced text classification and related tasks. Decoder-only models such as Llama exhibit strong performance an…
Exploring the Impact of Temperature on Large Language Models:Hot or Cold?
Lujun Li, Lama Sleem, Niccolo' Gentile +2
The sampling temperature, a critical hyperparameter in large language models (LLMs), modifies the logits before the softmax layer, thereby reshaping the distribution of output toke…
Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications
Lujun Li, Yiqun Wang, Radu State
Cloud cover in multispectral imagery (MSI) poses significant challenges for early season crop mapping, as it leads to missing or corrupted spectral information. Synthetic aperture…
Is Small Language Model the Silver Bullet to Low-Resource Languages Machine Translation?
Yewei Song, Lujun Li, Cedric Lothritz +6
Low-resource languages (LRLs) lack sufficient linguistic resources and are underrepresented in benchmark datasets, resulting in persistently lower translation quality than high-res…