4 papers
SENECA: Small-Sample Discrete Entropy Estimation via Self-Consistent Missing Mass
Lucas H. McCabe, H. Howie Huang
Discrete entropy estimation is a classic information theory problem, wherein the average information content of a discrete random variable is estimated from samples alone. Naive ap…
Estimating Semantic Alphabet Size for LLM Uncertainty Quantification
Lucas H. McCabe, Rimon Melamed, Thomas Hartvigsen +1
Many black-box techniques for quantifying the uncertainty of large language models (LLMs) rely on repeated LLM sampling, which can be computationally expensive. Therefore, practica…
Demystifying optimized prompts in language models
Rimon Melamed, Lucas H. McCabe, H. Howie Huang
Modern language models (LMs) are not robust to out-of-distribution inputs. Machine generated (``optimized'') prompts can be used to modulate LM outputs and induce specific behavior…
Prompts have evil twins
Rimon Melamed, Lucas H. McCabe, Tanay Wakhare +3
We discover that many natural-language prompts can be replaced by corresponding prompts that are unintelligible to humans but that provably elicit similar behavior in language mode…