ORQA: An Occupation-Realistic Question and Answer Framework for LLM Professional Knowledge

Shreyas Krishnan, Serina Chang, Abhishek Nagaraj

We present ORQA, a method for testing occupation-level knowledge in large language models. Prior methods either map abstract LLM skills to occupations via task definitions or utili…