most citedLarge Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

9 citations · 18 across the 8 of their papers we have counts for

collaborators

8 papers

cs.HC2024

Knowledge Acquisition and Integration with Expert-in-the-loop

Sajjadur Rahman, Frederick Choi, Hannah Kim +2

Constructing and serving knowledge graphs (KGs) is an iterative and human-centered process involving on-demand programming and analysis. In this paper, we present Kyurem, a program…

cs.CL20242 cited

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani +3

Remarkable performance of large language models (LLMs) in a variety of tasks brings forth many opportunities as well as challenges of utilizing them in production settings. Towards…

cs.LG2023

Knowledge Graphs are not Created Equal: Exploring the Properties and Structure of Real KGs

Nedelina Teneva, Estevam Hruschka

Despite the recent popularity of knowledge graph (KG) related tasks and benchmarks such as KG embeddings, link prediction, entity alignment and evaluation of the reasoning abilitie…

cs.CL2023

Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks

Pouya Pezeshkpour, Hayate Iso, Thom Lake +2

Numerous HR applications are centered around resumes and job descriptions. While they can benefit from advancements in NLP, particularly large language models, their real-world ado…

cs.CL20234 cited

Rethinking Language Models as Symbolic Knowledge Graphs

Vishwas Mruthyunjaya, Pouya Pezeshkpour, Estevam Hruschka +1

Symbolic knowledge graphs (KGs) play a pivotal role in knowledge-centric applications such as search, question answering and recommendation. As contemporary language models (LMs) t…

cs.CL20239 cited

Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Pouya Pezeshkpour, Estevam Hruschka

Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, a…