Presentation to Physiological Society, Cardiff '98 - similar to:- J. Physiol. 515P:48P

A.R. Gardner-Medwin, Dept. Physiology, University College London, London WC1E 6BT

Automated marking often employs a negative marking scheme that assigns a mark for a correct answer and an equal negative mark if the answer is wrong, with zero for an omitted answer. If there are multiple mutually exclusive possible answers (e.g. Maximal stimulation of the vagus leads to (a) profound tachycardia, (b) mild tachycardia, (c) mild bradycardia, (d) profound bradycardia, (e) no effect on the heart rate? ) then this discourages guessing since a student needs a >50% chance of being right to expect to gain from entering an answer. Tentative or seriously incomplete knowledge is (appropriately enough for a medical student) not rewarded. For example, a student who is fairly sure that the heart is slowed, but has no idea if this is brady- or tachy-cardia, would be unwise to hazard an answer.

Much of physiology is more amenable to simpler binary questions (e.g. ADH increases urine osmolarity. T/F?). The simple negative marking scheme is irrational when applied to such questions. Students are sometimes told that the logic is to discourage guessing, but this is not really true. Even a coin toss stands to give an average mark of zero, the same as skipping the questions. Students usually have some knowledge relevant to the issue. Even a tentative basis for preferring one answer over the other should lead on average to a benefit from putting it down. The system does not rationally discourage guessing unless a student believes that he/she has already got enough marks to pass and does not wish to take further risks.

The simple marking scheme can actually work to the disadvantage of stronger students. Suppose two students ‘A’ and ‘B’ have identical preferred answers to 100 T/F questions, 80 correct and 20 incorrect, with a net mark of 60%. Student A can identify 40 answers that he is correctly confident are right, while being uniformly unsure about the other 60. Student B is equally unsure about them all. Clearly student A has better insight and deserves a better assessment of knowledge related to these questions. Can he benefit from this? The only way he can use the extra knowledge in an exam would be to refrain from answering questions about which he is unsure (perhaps following the irrational advice dicussed above). This could drop his mark from 60% as low as 40%.

Confidence assessment on individual answers, combined with graded negative marking (Gardner-Medwin,1995), provides a more rational solution to this problem. It is more discerning in the assessment of students’ knowledge and during study and self-assessment it encourages students to to think about the basis, reliability and inter-relationships of their knowledge.

Software implementing confidence assessment is available from UCL (www.ucl.ac.uk/~cusplap).

Gardner-Medwin AR (1995) Association for Learning Technology Journal 3:80-85