activity
20232025
most citedPrometheus: Inducing Fine-grained Evaluation Capability in Language Models

15 citations · 18 across the 8 of their papers we have counts for

collaborators

8 papers

cs.AI2025

Parallel Key-Value Cache Fusion for Position Invariant RAG

Philhoon Oh, Jinwoo Shin, James Thorne

Recent advancements in Large Language Models (LLMs) underscore the necessity of Retrieval Augmented Generation (RAG) to leverage external information. However, LLMs are sensitive t…

cs.CL2024

Context Filtering with Reward Modeling in Question Answering

Sangryul Kim, James Thorne

Question Answering (QA) in NLP is the task of finding answers to a query within a relevant context retrieved by a retrieval system. Yet, the mix of relevant and irrelevant informat…

cs.CL2024★ 1 cited

CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean

Eunsu Kim, Juyoung Suk, Philhoon Oh +3

Despite the rapid development of large language models (LLMs) for the Korean language, there remains an obvious lack of benchmark datasets that test the requisite Korean cultural a…

cs.CL2024★ 2 cited

Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks

Minju Seo, Jinheon Baek, James Thorne +1

Despite large successes of recent language models on diverse tasks, they suffer from severe performance degeneration in low-resource settings with limited training data available.…

cs.IR2024

eXplainable Bayesian Multi-Perspective Generative Retrieval

EuiYul Song, Philhoon Oh, Sangryul Kim +1

Modern deterministic retrieval pipelines prioritize achieving state-of-the-art performance but often lack interpretability in decision-making. These models face challenges in asses…

cs.CL2023

Detrimental Contexts in Open-Domain Question Answering

Philhoon Oh, James Thorne

For knowledge intensive NLP tasks, it has been widely accepted that accessing more information is a contributing factor to improvements in the model's end-to-end performance. Howev…

cs.CL2023

Knowledge Corpus Error in Question Answering

Yejoon Lee, Philhoon Oh, James Thorne

Recent works in open-domain question answering (QA) have explored generating context passages from large language models (LLMs), replacing the traditional retrieval step in the QA…

cs.CL2023★ 15 cited

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

Seungone Kim, Jamin Shin, Yejin Cho +8

Recently, using a powerful proprietary Large Language Model (LLM) (e.g., GPT-4) as an evaluator for long-form responses has become the de facto standard. However, for practitioners…