1.8k citations · 3.2k across the 64 of their papers we have counts for
70 papers
Hyper3-CLIP: Hierarchy-Conditioned Hyperbolic Vision-Language Training
Matin Mahmood, Antonio Rueda-Toicen, Mohamed ElBassat +3
CLIP-like vision-language models (VLMs) trained with contrastive objectives learn strong global image-text representations, but their Euclidean embeddings and global pooling fail t…
Are We Recognizing the Jaguar or Its Background? A Diagnostic Framework for Jaguar Re-Identification
Antonio Rueda-Toicen, Abigail Allen Martin, Daniil Morozov +5
Jaguar re-identification (re-ID) from citizen-science imagery can look strong on standard retrieval metrics while still relying on the wrong evidence, such as background context or…
From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization
Ruangrin Ldallitsakool, Margarita Bugueño, Gerard de Melo
Recent NLP systems commonly represent documents as linear token sequences. Although this captures sequential order, it can hinder modeling long-range dependencies and global docume…
GorillaWatch: An Automated System for In-the-Wild Gorilla Re-Identification and Population Monitoring
Maximilian Schall, Felix Leonard Knöfel, Noah Elias König +11
Monitoring critically endangered western lowland gorillas is currently hampered by the immense manual effort required to re-identify individuals from vast archives of camera trap f…
Hierarchical Divide-and-Conquer for Fine-Grained Alignment in LLM-Based Medical Evaluation
Shunfan Zheng, Xiechi Zhang, Gerard de Melo +2
In the rapidly evolving landscape of large language models (LLMs) for medical applications, ensuring the reliability and accuracy of these models in clinical settings is paramount.…
NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning
Xin Yi, Shunfan Zheng, Linlin Wang +3
The emergence of finetuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate…
ACE-: Automatic Capability Evaluator for Multimodal Medical Models
Xiechi Zhang, Shunfan Zheng, Linlin Wang +4
As multimodal large language models (MLLMs) gain prominence in the medical field, the need for precise evaluation methods to assess their effectiveness has become critical. While b…
I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token
Roi Cohen, Konstantin Dobler, Eden Biran +1
Large Language Models are known to capture real-world knowledge, allowing them to excel in many downstream tasks. Despite recent advances, these models are still prone to what are…
Learning to Predict Usage Options of Product Reviews with LLM-Generated Labels
Leo Kohlenberg, Leonard Horns, Frederic Sadrieh +7
Annotating large datasets can be challenging. However, crowd-sourcing is often expensive and can lack quality, especially for non-trivial tasks. We propose a method of using LLMs a…
CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios
Zetian Ouyang, Yishuai Qiu, Linlin Wang +4
With the proliferation of Large Language Models (LLMs) in diverse domains, there is a particular need for unified evaluation standards in clinical medical scenarios, where models n…
Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough
Konstantin Dobler, Gerard de Melo
We investigate continued pretraining of LLMs for language adaptation on a tight academic budget: a setting in which only a few GPUs can be used in parallel, for a heavily constrain…
Mitigate the Gap: Investigating Approaches for Improving Cross-Modal Alignment in CLIP
Sedigheh Eslami, Gerard de Melo
Contrastive Language--Image Pre-training (CLIP) has manifested remarkable improvements in zero-shot classification and cross-modal vision-language tasks. Yet, from a geometrical po…
CommitBench: A Benchmark for Commit Message Generation
Maximilian Schall, Tamara Czinczoll, Gerard de Melo
Writing commit messages is a tedious daily task for many software developers, and often remains neglected. Automating this task has the potential to save time while ensuring that m…
On the Challenges and Opportunities in Generative AI
Laura Manduchi, Clara Meister, Kushagra Pandey +23
The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervi…
NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents
Tamara Czinczoll, Christoph Hönes, Maximilian Schall +1
While (large) language models have significantly improved over the last years, they still struggle to sensibly process long sequences found, e.g., in books, due to the quadratic sc…
MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
Yan Cai, Linlin Wang, Ye Wang +4
The emergence of various medical large language models (LLMs) in the medical domain has highlighted the need for unified evaluation standards, as manual evaluation of LLMs proves t…
Efficient Parallelization Layouts for Large-Scale Distributed Model Training
Johannes Hagemann, Samuel Weinbach, Konstantin Dobler +2
Efficiently training large language models requires parallelizing across hundreds of hardware accelerators and invoking various compute and memory optimizations. When combined, man…
IntentDial: An Intent Graph based Multi-Turn Dialogue System with Reasoning Path Visualization
Zengguang Hao, Jie Zhang, Binxia Xu +3
Intent detection and identification from multi-turn dialogue has become a widely explored technique in conversational agents, for example, voice assistants and intelligent customer…
FARSEC: A Reproducible Framework for Automatic Real-Time Vehicle Speed Estimation Using Traffic Cameras
Lucas Liebe, Franz Sauerwald, Sylwester Sawicki +6
Estimating the speed of vehicles using traffic cameras is a crucial task for traffic surveillance and management, enabling more optimal traffic flow, improved road safety, and lowe…
Large Language Models and Knowledge Graphs: Opportunities and Challenges
Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo +13
Large Language Models (LLMs) have taken Knowledge Representation -- and the world -- by storm. This inflection point marks a shift from explicit knowledge representation to a renew…
Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?
Margarita Bugueño, Gerard de Melo
Given the success of Graph Neural Networks (GNNs) for structure-aware machine learning, many studies have explored their use for text classification, but mostly in specific domains…
MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models
Sepehr Janghorbani, Gerard de Melo
Recent breakthroughs in self supervised training have led to a new class of pretrained vision language models. While there have been investigations of bias in multimodal models, th…
Democratization of Retail Trading: Can Reddit's WallStreetBets Outperform Investment Bank Analysts?
Tolga Buz, Gerard de Melo
The recent hype around Reddit's WallStreetBets (WSB) community has inspired research on its impact on our economy and society. Still, one important question remains: Can WSB's comm…
ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities
Terry Yue Zhuo, Yaqing Liao, Yuecheng Lei +5
We introduce ViLPAct, a novel vision-language benchmark for human activity planning. It is designed for a task where embodied AI agents can reason and forecast future actions of hu…