62 citations · 146 across the 49 of their papers we have counts for
58 papers
Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models
Xiaoyu Luo, Tao Ren, Wenrui Yu +3
The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems…
SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices
Ernests Lavrinovics, Marco Letizia, Roy Janco +3
We present SigmaScale, a method for learning auxiliary scaling matrices to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression. Instea…
Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls
Rasmus T. Aavang, Rasmus Tjalk-Bøggild, Alexandre Iolov +3
Earnings calls are a key source of financial information about public companies. However, extracting information from these calls is difficult. Unlike the templatic filings require…
When Discourse Pressures Conflict: Information Structure in Vision-Language Model Outputs
Marcell Fekete, Johannes Bjerva, Tamás Káldi
Vision-language models (VLMs) are increasingly evaluated for whether they identify the right visual content, but little is known about whether they express such content in a discou…
Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects
Xiaoyu Luo, Wenrui Yu, Qiongxiu Li +1
Autoregressive language models (ARMs) have been shown to memorize and occasionally reproduce training data verbatim, raising concerns about privacy and copyright liability. Diffusi…
Semantic Leakage from Image Embeddings
Yiyi Chen, Qiongkai Xu, Desmond Elliott +2
Image embeddings are generally assumed to pose limited privacy risk. We challenge this assumption by formalizing semantic leakage as the ability to recover semantic structures from…
Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework
Xiaoyu Luo, Yiyi Chen, Qiongxiu Li +1
Large Language Models (LLMs) have been reported to "leak" Personally Identifiable Information (PII), with successful PII reconstruction often interpreted as evidence of memorizatio…
Limited-Resource Adapters Are Regularizers, Not Linguists
Marcell Fekete, Nathaniel R. Robinson, Ernests Lavrinovics +4
Cross-lingual transfer from related high-resource languages is a well-established strategy to enhance low-resource language technologies. Prior work has shown that adapters show pr…
LAGO: Few-shot Crosslingual Embedding Inversion Attacks via Language Similarity-Aware Graph Optimization
Wenrui Yu, Yiyi Chen, Johannes Bjerva +2
We propose LAGO - Language Similarity-Aware Graph Optimization - a novel approach for few-shot cross-lingual embedding inversion attacks, addressing critical privacy vulnerabilitie…
Shared Path: Unraveling Memorization in Multilingual LLMs through Language Similarities
Xiaoyu Luo, Yiyi Chen, Johannes Bjerva +1
We present the first comprehensive study of Memorization in Multilingual Large Language Models (MLLMs), analyzing 95 languages using models across diverse model scales, architectur…
MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations
Ernests Lavrinovics, Russa Biswas, Katja Hose +1
Large Language Models (LLMs) have inherent limitations of faithfulness and factuality, commonly referred to as hallucinations. Several benchmarks have been developed that provide a…
Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality
Mike Zhang, Johannes Bjerva, Russa Biswas
We introduce fs1, a simple yet effective method that improves the factuality of reasoning traces by collecting them from large reasoning models and grounding them in knowledge grap…
NLP Security and Ethics, in the Wild
Heather Lent, Erick Galinkin, Yiyi Chen +3
As NLP models are used by a growing number of end-users, an area of increasing importance is NLP Security (NLPSec): assessing the vulnerability of models to malicious attacks and d…
On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation
Rune Birkmose, Nathan Mørkeberg Reece, Esben Hofstedt Norvin +2
This paper investigates whether Large Language Models (LLMs), fine-tuned on synthetic but domain-representative data, can perform the twofold task of (i) slot and intent detection…
Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond
Qiongxiu Li, Xiaoyu Luo, Yiyi Chen +1
The role of memorization in machine learning (ML) has garnered significant attention, particularly as modern models are empirically observed to memorize fragments of training data.…
How Do Hackathons Foster Creativity? Towards AI Collaborative Evaluation of Creativity at Scale
Jeanette Falk, Yiyi Chen, Janet Rafner +3
Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes a…
ALGEN: Few-shot Inversion Attacks on Textual Embeddings using Alignment and Generation
Yiyi Chen, Qiongkai Xu, Johannes Bjerva
With the growing popularity of Large Language Models (LLMs) and vector databases, private textual data is increasingly processed and stored as numerical embeddings. However, recent…
HiFi-KPI: A Dataset for Hierarchical KPI Extraction from Earnings Filings
Rasmus T. Aavang, Giovanni Rizzi, Rasmus Tjalk-Bøggild +3
Accurate tagging of earnings reports can yield significant short-term returns for stakeholders. The machine-readable inline eXtensible Business Reporting Language (iXBRL) is mandat…
SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agents
Mike Zhang, Amalie Pernille Dilling, Léon Gondelman +3
Providing high-quality feedback on student assignments is crucial for student success, but it is heavily limited by time and budgetary constraints. In this work, we introduce Synth…
Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective
Ernests Lavrinovics, Russa Biswas, Johannes Bjerva +1
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) based applications including automated text generation, question answering, chatbots, and others.…
How Good is Your Wikipedia? Auditing Data Quality for Low-resource and Multilingual NLP
Kushal Tatariya, Artur Kulmizev, Wessel Poelman +6
Wikipedia's perceived high quality and broad language coverage have established it as a fundamental resource in NLP. However, in recent years, such assumptions of high quality have…
Linguistically Grounded Analysis of Language Models using Shapley Head Values
Marcell Fekete, Johannes Bjerva
Understanding how linguistic knowledge is encoded in language models is crucial for improving their generalisation capabilities. In this paper, we investigate the processing of mor…
Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis
Yiyi Chen, Qiongxiu Li, Russa Biswas +1
Language Confusion is a phenomenon where Large Language Models (LLMs) generate text that is neither in the desired language, nor in a contextually appropriate language. This phenom…
Against All Odds: Overcoming Typology, Script, and Language Confusion in Multilingual Embedding Inversion Attacks
Yiyi Chen, Russa Biswas, Heather Lent +1
Large Language Models (LLMs) are susceptible to malicious influence by cyber attackers through intrusions such as adversarial, backdoor, and embedding inversion attacks. In respons…