TITLE:
Towards More Efficient Assessments: Increasing Information from Objective Examinations
AUTHORS:
Alan E. Dugdale
KEYWORDS:
Objective Assessment; Information
JOURNAL NAME:
Creative Education,
Vol.4 No.6A,
June
14,
2013
ABSTRACT:
Objective
examination questions are widely used to assess students’ knowledge, but the standard MCQ with a stem followed by 4 - 6 possible answers one of which is chosen as correct is very inefficient. Simple
changes to the format can treble the information gained. Information theory is
the tool for assessing the information content of electronic and other
communications. The “bit” is the unit of information and equals on true/false
choice. I have applied basic information theory to objective questions. The
standard MCQ with a stem, a choice of 4 possible answers (mark one answer true)
and an expected correct answer rate of 70% yields 1.36 bits. A MTFQ with 4
choices where the student must answer true/false to every possible answer gives
3.52 bits of information. By adding a “don’t know” option the
same MTFQ gives 4.72 bits of information, 350% of the standard MCQ. Thirty MTFQ
with don’t know give the same information about students’ knowledge as 100 standard MCQs. The effort needed to set, sit and mark is the
same for both. Small changes to the format of objective
questions give large gains in efficiency. We should balance these gains against
possible disadvantages.