Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code

Jaromir Savelka, Arav Agarwal, Christopher Bogart +1

We analyzed effectiveness of three generative pre-trained transformer (GPT) models in answering multiple-choice question (MCQ) assessments, often involving short snippets of code,…