The Examination of Accreditation for Foreign Medical Graduates in Greece: an Evaluation of the Multiple Choice Questions Format Using the Difficulty and Discrimination Indices

Abstract

Background: The multiple choice questions (MCQ) format is the most commonly used written assessment technique for the accreditation of foreign medical graduates in Greece.

Objective: To evaluate the appropriateness of the range of the MCQ tests and to compare the performance among foreign medical graduates, Greek medical students and interns (medical doctors during their residency).

Methods: Twenty-six items from the internal medicine question paper and 24 from the surgery question paper were randomly selected from an MCQ format used in one assessment. For these items discrimination and difficulty indices were calculated for separate groups of candidates and volunteer participants (medical students and interns). Comparisons were made between group scores, first considering the whole questionnaire as a single entity and then using scores for each discipline calculated separately.

Results: A significant number of “inappropriate” questions were included in the examination. Surgery questions were more candidate-oriented, given the best range of acceptable Difficulty Index values for that group of participants while internal medicine questions proved to be more appropriate for medical students. Furthermore, comparisons of groups performed using a total score over the whole range of the two disciplines revealed a significantly better performance of interns over students (p<0.001), whereas comparisons performed separately for each discipline revealed no significant difference between interns and students in surgery scores but a significant difference in internal medicine scores (p<0.001).

Conclusion: These findings suggest the importance of the evaluation of the MCQs before using them in examinations, aiming at revising inappropriate questions. In order to evaluate the participants’ performance, calculation of scores across separate disciplines is proposed, since it is less likely to be biased towards good performance in the questions of one discipline.