9 papers
Language Equality has a Price: A Systematic Investigation of Multi-turn LLM Performance for EU-24+
Sherzod Hakimov, Karl Osswald, Jelle Psurek +3
We evaluate large language models (LLMs) as language agents playing goal-directed dialogue games in self-play across 30 languages: the 24 official EU languages plus six others. Unl…
GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines
Jafar Isbarov, Umid Suleymanov, Ilia Shumailov +1
AI-powered agents are increasingly embedded in continuous integration and continuous delivery/deployment (CI/CD) pipelines to autonomously review pull requests (PRs), triage issues…
CASPIAN: Online Detection and Attribution of Cascade Attacks in LLM Multi-Agent Systems via Cross-Channel Causal Monitoring
Kavana Venkatesh, Jafar Isbarov, Saad Amin +2
Cascade attacks in LLM multi-agent systems (MAS) arise when adversarial influence propagates across agents and leads to escalated system-level failures through complex agent intera…
Bypassing AI Control Protocols via Agent-as-a-Proxy Attacks
Jafar Isbarov, Murat Kantarcioglu
As AI agents automate critical workloads, they remain vulnerable to indirect prompt injection (IPI) attacks. Current defenses rely on monitoring protocols that jointly evaluate an…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
TUMLU: A Unified and Native Language Understanding Benchmark for Turkic Languages
Jafar Isbarov, Arofat Akhundjanova, Mammad Hajili +13
Being able to thoroughly assess massive multi-task language understanding (MMLU) capabilities is essential for advancing the applicability of multilingual language models. However,…
Comparison of algorithms used in single-cell transcriptomic data analysis
Jafar Isbarov, Elmir Mahammadov
Single-cell analysis is an increasingly relevant approach in "omics'' studies. In the last decade, it has been applied to various fields, including cancer biology, neuroscience, an…
Enhanced document retrieval with topic embeddings
Kavsar Huseynova, Jafar Isbarov
Document retrieval systems have experienced a revitalized interest with the advent of retrieval-augmented generation (RAG). RAG architecture offers a lower hallucination rate than…
Open foundation models for Azerbaijani language
Jafar Isbarov, Kavsar Huseynova, Elvin Mammadov +2
The emergence of multilingual large language models has enabled the development of language understanding and generation systems in Azerbaijani. However, most of the production-gra…