27 citations · 81 across the 33 of their papers we have counts for
7 papers · 2 filters
The Canonical Order Problem: When Large Language Models Are Unreliable Knowledge Bases for Multi-Valued Relations
Timo Pierre Schrader, Annemarie Friedrich, Simon Razniewski +1
Large language models (LLMs) are increasingly used as knowledge bases (KBs) due to the vast amount of knowledge they acquire during pre-training. While many works focus on extracti…
GPTKB 2.0: Browsing, Querying, and Auditing a Disambiguated LLM-Derived Knowledge Base
Yujia Hu, Tuan-Phong Nguyen, Simon Razniewski
We present a web demo for exploring a large-scale disambiguated knowledge base (KB) materialized from a large language model (LLM). GPTKB 2.0 contains 38.4M triples over 1.6M canon…
Constructing Disambiguated Knowledge Bases from Large Language Models at Scale
Yujia Hu, Tuan-Phong Nguyen, Simon Razniewski
Automated Knowledge Base Construction (AKBC) is a core NLP task, and recent work proposes generating knowledge bases directly from large language models (LLMs), treating the model…
LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs
Muhammed Saeed, Simon Razniewski
Flagship language models appear saturated on benchmarks like MMLU (Hendrycks et al., 2021), scoring above 90% - yet benchmarks test only what the experimenter thought to ask, the a…
Is It Novel and Why? Fine-Grained Patent Novelty Prediction Based on Passage Retrieval
Valentin Knappich, Anna Hätty, Simon Razniewski +1
Novelty assessment is a critical yet complex task in the examination process for patent acceptance, requiring examiners to determine whether an invention is disclosed in a prior ar…
Beyond Questions: Evaluating the Stated Knowledge of LLMs
Luca Giordano, Simon Razniewski
Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined qu…
LLMpedia: A Transparent Framework to Materialize an LLM's Encyclopedic Knowledge at Scale
Muhammed Saeed, Simon Razniewski
Benchmarks like MMLU suggest flagship language models approach factuality saturation above 90\%. \emph{LLMpedia} shows this picture is incomplete. We materialize 1.3M encyc…