4 papers
Text Corpora as Concept Fields: Black-Box Hallucination and Novelty Measurement
Nicholas S. Kersting, Vittorio Castelli, Chieh Ting Yeh +3
We introduce the \textbf{Concept Field} of a text corpus: a local drift field with pointwise uncertainty, estimated in sentence-embedding space from the deltas between consecutive…
Evaluating Quality of Answers for Retrieval-Augmented Generation: A Strong LLM Is All You Need
Yang Wang, Alberto Garcia Hernandez, Roman Kyslyi +1
We present a comprehensive study of answer quality evaluation in Retrieval-Augmented Generation (RAG) applications using vRAG-Eval, a novel grading system that is designed to asses…
Harmonic LLMs are Trustworthy
Nicholas S. Kersting, Mohammad Rahman, Suchismitha Vedala +1
We introduce an intuitive method to test the robustness (stability and explainability) of any black-box LLM in real-time via its local deviation from harmoniticity, denoted as …
Harmonic Machine Learning Models are Robust
Nicholas S. Kersting, Yi Li, Aman Mohanty +2
We introduce Harmonic Robustness, a powerful and intuitive method to test the robustness of any machine-learning model either during training or in black-box real-time inference mo…