8 papers
When Skills Don't Help: A Negative Result on Procedural Knowledge for Tool-Grounded Agents in Offensive Cybersecurity
Samuel Jacob Chacko, James Hugglestone, Chashi Mahiul Islam +1
Agent Skills, structured packages of procedural knowledge loaded into an LLM agent at inference time, are widely reported to improve task pass rates by an average of 16.2~percentag…
Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
Chashi Mahiul Islam, Alan Villarreal, Mao Nishino +2
As Large Language Models (LLMs) are increasingly integrated into agentic workflows, their unpredictability stemming from numerical instability has emerged as a critical reliability…
Spatial-ViLT: Enhancing Visual Spatial Reasoning through Multi-Task Learning
Chashi Mahiul Islam, Oteo Mamo, Samuel Jacob Chacko +2
Vision-language models (VLMs) have advanced multimodal reasoning but still face challenges in spatial reasoning for 3D scenes and complex object configurations. To address this, we…
Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging
Montasir Shams, Chashi Mahiul Islam, Shaeke Salman +2
Vision transformers (ViTs) have rapidly gained prominence in medical imaging tasks such as disease classification, segmentation, and detection due to their superior accuracy compar…
DeepSeek on a Trip: Inducing Targeted Visual Hallucinations via Representation Vulnerabilities
Chashi Mahiul Islam, Samuel Jacob Chacko, Preston Horne +1
Multimodal Large Language Models (MLLMs) represent the cutting edge of AI technology, with DeepSeek models emerging as a leading open-source alternative offering competitive perfor…
Mechanistic Understandings of Representation Vulnerabilities and Engineering Robust Vision Transformers
Chashi Mahiul Islam, Samuel Jacob Chacko, Mao Nishino +1
While transformer-based models dominate NLP and vision applications, their underlying mechanisms to map the input space to the label space semantically are not well understood. In…
Adversarial Attacks on Large Language Models Using Regularized Relaxation
Samuel Jacob Chacko, Sajib Biswas, Chashi Mahiul Islam +2
As powerful Large Language Models (LLMs) are now widely used for numerous practical applications, their safety is of critical importance. While alignment techniques have significan…
Malicious Path Manipulations via Exploitation of Representation Vulnerabilities of Vision-Language Navigation Systems
Chashi Mahiul Islam, Shaeke Salman, Montasir Shams +2
Building on the unprecedented capabilities of large language models for command understanding and zero-shot recognition of multi-modal vision-language transformers, visual language…