Certainty-Based Marking (CBM) in Moodle: History & Options
Certainty-Based Marking rewards students for distinguishing between uncertain and reliable answers and thereby encourages deeper thinking about issues surrounding each question. The principles and outcomes are set out in my teaching publications. The system (LAPT) that I devised in 1994 for physiology and maths teaching at UCL, has now evolved to a more flexible system, purpose-designed for CBM Selftests.
Moodle is an open-access Learning Management System, widely used in many contexts. Since 2007, I have provided code modifications for current Moodle versions, available for download. They are simply copied into an installation and can be subsequently restored if required. There have been no reported adverse events with any of these downloads, and they don't affect function when CBM is not in use. Moodle documentation for CBM is available on the official Moodle site. Since 2012 (versions 2.6, 2.7) standard Moodle has included an interface incorporating student feedback for CBM. An additional PLUGIN is still required for data reports for staff, and some additional code modifications improve functionality and feedback for students, particularly if CBM has been turned on and off during the life of a quiz.
Specific Features for different Moodle versions
Possible improvements in CBM that could still improve Moodle 2.6+ (most included with code modifications in earlier versions) :
- Moodle 1.9-2.0: CBM behaviour is only available with code modifications. All questions are assigned equal weight with CBM behaviour, giving marks 1, 2, or 3 if correct at C=1,2,3. A 'No Idea' option is included (mark=0) so students don't have to guess to see feedback if they have no idea. Submission of answers is with a single click. Percentage scores are optionally shown relative to the questions attempted. Attempt reports only give feedback for Qs the student has attempted. A 'Knowledge' score is included, which is accuracy scaled so pure guesses (at C=1) would give on average 0%.
- Moodle 2.1: Basic CBM interface behaviour was implemented in standard Moodle code, but CBM users are advised to upgrade from this version since proper CBM feedback is not available.
- Moodle 2.2-2.5: CBM gives marks 0.33, 0.67, or 1 for C=1,2,3 unless weights are all changed to 3. This is changed with code modifications so that all questions get marks 1,2,3 if correct. A 'No Idea' option is implemented with the patches. Feedback (with the modifications) includes both accuracy and CBM scores, with breakdown of performance by C level - as with Moodle 1.9-2.0.
- Moodle 2.6-2.7: The CBM interface and feedback to students were improved in core Moodle, and now work well. Default weighting of Qs gives marks 1,2,3 if correct and 0,-2,-6 if wrong, for C=1,2,3. Question weights ('max marks') can be varied to affect these marks in proportion, but this is not recommended to avoid confusion. NB CBM Grade Reports for teachers require installation of a PLUGIN, from the Moodle site, or from the download site here.
- Include a 'Don't Know' or 'No Idea' option, so students aren't forced to guess to receive feedback.
- Provide feedback (answers, explanations) only for Qs a student has attempted, not for all. Important for self-testing.
- Arrange the interface so submission of an answer can be done with one click (the chosen certainty), without need for a second 'enter' click.
- Show 'Knowledge' scores, scaled so that pure guesses would on average give a zero score.
- [Would improve all versions] Show the Moodle 'Grade' in gradebook summaries etc. as the CB Grade (reflecting CB Accuracy, up to 100%, which is the best comparison to make with conventional grades) rather than as the total of CB Marks (up to 300% of the maximum without CBM).
Recent forum discussions about CBM in Moodle
- In Moodle 2.6+, it is usually best to leave the weighting (or "maximum mark") for each question at its default value (=1) so that standard CBM scores (1,2,3, etc.) apply. These are straightforwardly interpreted by students and avoid confusion - for example whether a mark of 3 was for a confident answer at C=3, or an unconfident answer at C=1 on a question assigned triple weight.
- It can be confusing if, in a quiz with CBM, you include questions with multiple parts. The student may be sure about some parts and unsure about others. Individual parts are best asked separately with CBM, or else placed in a separate type of quiz that doesn't use CBM.
- At present, Moodle "Grades" - based on total marks - can be up to 300% with CBM. It is important to ensure that the option 'Unlimited Grades' is switched on under Site Administration -> Grades -> General settings. Otherwise, even very modest grades with CBM may show in some gradebook summary screens as 100%. There seems no evident situation in which it would be better to have this switch turned off.
- The CB Grade, shown in reviews of individual attempts and CBM Grade summaries, gives a proper comparison with conventional Grades (just based on accuracy). It is probably the best single number to use in student assessment, though not currently shown as the "Moodle Grade".
Certainty Based Marking in Moodle (Isabelle Langeveld's blog) 2011
Marks, accuracy and CBM, 2013
Removing CBM bonus/penalty from a quiz grade, 2014
CBM - Change Penalties, 2014
Truncation of Grades in the Gradebook, 2014
Tony Gardner-Medwin (UCL)
Revised August 2014