Abstract for Physiological Society Teaching Workshop, Bristol, July
Confidence-Based Marking: encouraging rigour through assessment
Dept. Physiology, University College London, London WC1E 6BT
Students, especially medical students, have a lot to learn and a lot of
assessments. Many able students find that they can do well without much
thought - since the first idea they think of in answer to a question
usually has a good chance of being correct, and this approach can get
them good marks in exams. This tends to reward rote-learning, and a
superficial approach. Confidence-based marking (CBM), in which a
student must indicate confidence in each answer and be graded according
to a properly motivating mark scheme, helps to encourage reflection,
justification and rigour. It rewards both justification to the point of
high confidence and the ability to identify reasons for reservation
about an answer, and it therefore encourages a more rigorous approach
both to learning and assessment. Experience at UCL and Imperial College
over many years has shown that students find the concept and our
marking scheme easy to understand, fair, and a stimulus to learning.
Our dissemination programme is designed to encourage uptake in other
universities and other disciplines, wherever students encounter
questions to which the answers can be marked as right or wrong.
Our scheme for CBM is simple: confidence is rated 1, 2 or 3 and the
marks awarded for correct answers are the same: 1, 2 or 3. Incorrect
answers receive penalties of 0, -2 or -6. This graded negative marking
rewards a student who can discriminate reliable from uncertain
knowledge. Highest marks are obtained by choosing C=1 if the
probability of being correct is <67% and C=3 if it is >80%. The
features that students appreciate are that it correctly distinguishes
sound knowledge from a lucky guess, and it deservedly penalises
confident misconceptions more than ignorance. In summative assessments
it has greater reliability and validity than marking based simply on
the numbers of correct answers (Gardner-Medwin & Gahan, 2003). It
improves signal-to-noise ratios by reducing the weighting of answers
based on uncertain knowledge, which are associated with high variance.
It shows no evidence for gender bias in practised students -- a concern
sometimes volunteered by people on first hearing of CBM. CBM avoids a
serious hazard that arises with use of a conventional fixed negative
marking scheme (+/-1) with true/false questions, which can disadvantage
students who have the insight to see when there are reasons for
reservation, or who are simply diffident, or who take too literally
advice that they should refrain from guessing (Gardner-Medwin, 1999).
There are simple arrangements for staff and students in new
institutions to experience CBM in practice and to develop, adapt and
run existing exercises based on their own material. The software can
either be used via the UCL website (www.ucl.ac.uk/lapt) or can be
copied elsewhere. For summative tests requiring invigilation we prefer
to use Optical Mark Reader technology, for which cards implementing CBM
for either True/False or multiple choice (pick one from A-E) questions
are available from UCL and can currently be processed by UCL. Other
question types (e.g. extended matching sets) can be handled with cards
available from Speedwell Computing Services (www.speedwell.co.uk).
Supported by HEFCE through the Fund for the Development of Teaching and
Learning, Phase 4.
Gardner-Medwin A.R. & Gahan M. (2003) Formative and Summative
Confidence-Based Assessment. Seventh International Computer-Aided
Assessment Conference Proceedings, Loughborough University, UK, pp.
147-155 ( www.caaconference.com )
Gardner-Medwin, A.R. (1999) Rational and irrational marking
schemes. Journal of Physiology, 515P: 48P