7 citations · 43 across the 33 of their papers we have counts for
36 papers
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls
Rasmus T. Aavang, Rasmus Tjalk-Bøggild, Alexandre Iolov +3
Earnings calls are a key source of financial information about public companies. However, extracting information from these calls is difficult. Unlike the templatic filings require…
CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
Mike Zhang, Ali Basirat, Desmond Elliott
Prior work establishes that controlled contrastiveness between self-generated responses from large language models, set via reward scores, improves downstream preference tuning in…
WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain
Matthias De Lange, Warre Veys, Federico Retyk +16
Today's evolving labor markets rely increasingly on recommender systems for hiring, talent management, and workforce analytics, with natural language processing (NLP) capabilities…
UniSkill: A Dataset for Matching University Curricula to Professional Competencies
Nurlan Musazade, Joszef Mezei, Mike Zhang
Skill extraction and recommendation systems have been studied from recruiter, applicant, and education perspectives. While AI applications in job advertisements have received broad…
Do Large Language Models Adapt to Language Variation across Socioeconomic Status?
Elisa Bassignana, Mike Zhang, Dirk Hovy +1
Humans adjust their linguistic style to the audience they are addressing. However, the extent to which LLMs adapt to different social contexts is largely unknown. As these models i…
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…
NLPnorth @ TalentCLEF 2025: Comparing Discriminative, Contrastive, and Prompt-Based Methods for Job Title and Skill Matching
Mike Zhang, Rob van der Goot
Matching job titles is a highly relevant task in the computational job market domain, as it improves e.g., automatic candidate matching, career path prediction, and job market anal…
Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality
Mike Zhang, Johannes Bjerva, Russa Biswas
We introduce fs1, a simple yet effective method that improves the factuality of reasoning traces by collecting them from large reasoning models and grounding them in knowledge grap…
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam +42
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While mu…
On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation
Rune Birkmose, Nathan Mørkeberg Reece, Esben Hofstedt Norvin +2
This paper investigates whether Large Language Models (LLMs), fine-tuned on synthetic but domain-representative data, can perform the twofold task of (i) slot and intent detection…
How Do Hackathons Foster Creativity? Towards AI Collaborative Evaluation of Creativity at Scale
Jeanette Falk, Yiyi Chen, Janet Rafner +3
Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes a…
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Longxu Dou, Qian Liu, Fan Zhou +38
Sailor2 is a family of cutting-edge multilingual language models for South-East Asian (SEA) languages, available in 1B, 8B, and 20B sizes to suit diverse applications. Building on…
HiFi-KPI: A Dataset for Hierarchical KPI Extraction from Earnings Filings
Rasmus T. Aavang, Giovanni Rizzi, Rasmus Tjalk-Bøggild +3
Accurate tagging of earnings reports can yield significant short-term returns for stakeholders. The machine-readable inline eXtensible Business Reporting Language (iXBRL) is mandat…
SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agents
Mike Zhang, Amalie Pernille Dilling, Léon Gondelman +3
Providing high-quality feedback on student assignments is crucial for student success, but it is heavily limited by time and budgetary constraints. In this work, we introduce Synth…
SnakModel: Lessons Learned from Training an Open Danish Large Language Model
Mike Zhang, Max Müller-Eberstein, Elisa Bassignana +1
We present SnakModel, a Danish large language model (LLM) based on Llama2-7B, which we continuously pre-train on 13.6B Danish words, and further tune on 3.7M Danish instructions. A…
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Angelika Romanou, Negar Foroutan, Anna Sotnikova +56
The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal val…
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana +66
Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cul…
Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturers
Mike Zhang, Euan D Lindsay, Frederik Bode Thorbensen +2
End of semester student evaluations of teaching are the dominant mechanism for providing feedback to academics on their teaching practice. For large classes, however, the volume of…
Computational Job Market Analysis with Natural Language Processing
Mike Zhang
[Abridged Abstract] Recent technological advances underscore labor market dynamics, yielding significant consequences for employment prospects and increasing job vacancy data acros…
Can Humans Identify Domains?
Maria Barrett, Max Müller-Eberstein, Elisa Bassignana +3
Textual domain is a crucial property within the Natural Language Processing (NLP) community due to its effects on downstream model performance. The concept itself is, however, loos…
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Shivalika Singh, Freddie Vargus, Daniel Dsouza +30
Datasets are foundational to many breakthroughs in modern artificial intelligence. Many recent achievements in the space of natural language processing (NLP) can be attributed to t…
Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings
Elena Senger, Mike Zhang, Rob van der Goot +1
Recent years have brought significant advances to Natural Language Processing (NLP), which enabled fast progress in the field of computational job market analysis. Core tasks in th…
Rethinking Skill Extraction in the Job Market Domain using Large Language Models
Khanh Cao Nguyen, Mike Zhang, Syrielle Montariol +1
Skill Extraction involves identifying skills and qualifications mentioned in documents such as job postings and resumes. The task is commonly tackled by training supervised models…