Abstract for Physiological Society Teaching Workshop, Bristol, July 2005

Confidence-Based Marking: encouraging rigour through assessment
A.R. Gardner-Medwin
Dept. Physiology, University College London, London WC1E 6BT

Students, especially medical students, have a lot to learn and a lot of assessments. Many able students find that they can do well without much thought - since the first idea they think of in answer to a question usually has a good chance of being correct, and this approach can get them good marks in exams. This tends to reward rote-learning, and a superficial approach. Confidence-based marking (CBM), in which a student must indicate confidence in each answer and be graded according to a properly motivating mark scheme, helps to encourage reflection, justification and rigour. It rewards both justification to the point of high confidence and the ability to identify reasons for reservation about an answer, and it therefore encourages a more rigorous approach both to learning and assessment. Experience at UCL and Imperial College over many years has shown that students find the concept and our marking scheme easy to understand, fair, and a stimulus to learning. Our dissemination programme is designed to encourage uptake in other universities and other disciplines, wherever students encounter questions to which the answers can be marked as right or wrong.

Our scheme for CBM is simple: confidence is rated 1, 2 or 3 and the marks awarded for correct answers are the same: 1, 2 or 3. Incorrect answers receive penalties of 0, -2 or -6. This graded negative marking rewards a student who can discriminate reliable from uncertain knowledge. Highest marks are obtained by choosing C=1 if the probability of being correct is <67% and C=3 if it is >80%. The features that students appreciate are that it correctly distinguishes sound knowledge from a lucky guess, and it deservedly penalises confident misconceptions more than ignorance. In summative assessments it has greater reliability and validity than marking based simply on the numbers of correct answers (Gardner-Medwin & Gahan, 2003). It improves signal-to-noise ratios by reducing the weighting of answers based on uncertain knowledge, which are associated with high variance. It shows no evidence for gender bias in practised students -- a concern sometimes volunteered by people on first hearing of CBM. CBM avoids a serious hazard that arises with use of a conventional fixed negative marking scheme (+/-1) with true/false questions, which can disadvantage students who have the insight to see when there are reasons for reservation, or who are simply diffident, or who take too literally advice that they should refrain from guessing (Gardner-Medwin, 1999).

There are simple arrangements for staff and students in new institutions to experience CBM in practice and to develop, adapt and run existing exercises based on their own material. The software can either be used via the UCL website (www.ucl.ac.uk/lapt) or can be copied elsewhere. For summative tests requiring invigilation we prefer to use Optical Mark Reader technology, for which cards implementing CBM for either True/False or multiple choice (pick one from A-E) questions are available from UCL and can currently be processed by UCL. Other question types (e.g. extended matching sets) can be handled with cards available from Speedwell Computing Services (www.speedwell.co.uk).
Supported by HEFCE through the Fund for the Development of Teaching and Learning, Phase 4.

Gardner-Medwin A.R. & Gahan M. (2003) Formative and Summative Confidence-Based Assessment. Seventh International Computer-Aided Assessment Conference Proceedings, Loughborough University, UK, pp. 147-155 ( www.caaconference.com )
Gardner-Medwin, A.R. (1999) Rational and irrational marking schemes.  Journal of Physiology, 515P: 48P