Abstract of a presentation at Innovation 2008, April 14-15, Breckenridge CO, USA
( Listen to webcast of this paper,   Download Power Point Presentation )

Certainty-Based Marking: rewarding good judgment of what is or is not reliable
Tony Gardner-Medwin
Dept. Physiology, UCL, London WC1E 6BT

Abstract:
Certainty-Based Marking (CBM) scores an objective test (usually on a computer) in a way that rewards students for identifying and distinguishing when individual answers are reliable or unreliable. It penalises confident errors and rewards a thoughtful and realistic appraisal by the student of the basis and limitations of his/her knowledge. It has long been known to stimulate learning and improve assessment, but is nevertheless little used in schools or universities. Why not? Before the computer age there were technical barriers. Some forms of CBM have been too complex, or have failed to motivate true reporting of uncertainty. Also, teachers have sometimes confusingly thought CBM rewards self-confidence, which it does not : it rewards reliable answers, and good judgment of reliability.

Nowadays CBM is not only easily implemented (see e.g. www.ucl.ac.uk/lapt), but all the more essential because of the easy access to information on the internet. This has turned the role of knowledge assessment somewhat on its head. Lack of knowledge is no longer such a problem : you can look it up. What is important is to acknowledge uncertainty, and to eradicate confident misconceptions that lead to complacency, confusion and danger. Though information is still crucial, education is more about handling information than having information. For students who do well in objective tests without much thinking - perhaps even without understanding what they have learned - CBM raises the stakes. It forces them to think more about how their knowledge knits together - whether it forms the consistent web that we call understanding.

Experience in London with a simple form of CBM has proved popular with students in their study and beneficial to student-teacher interaction. Students need practice to become familiar with the mark scheme, but it rewards realism in a pedagogically sound manner and improves the statistical quality of exam data by marking unconfident answers without penalty but with reduced weight. It avoids two of the cardinal crimes of assessment: rewarding lucky guesses as if they were knowledge, and treating confident misconceptions as no worse than acknowledged ignorance. CBM deserves a much bigger place in the future.