Certainty-Based Marking (CBM) in Moodle: History & Options

• Moodle 1.9-2.0: CBM behaviour is only available with code modifications. All questions are assigned equal weight with CBM behaviour, giving marks 1, 2, or 3 if correct at C=1,2,3. A 'No Idea' option is included (mark=0) so students don't have to guess to see feedback if they have no idea. Submission of answers is with a single click. Percentage scores are optionally shown relative to the questions attempted. Attempt reports only give feedback for Qs the student has attempted. A 'Knowledge' score is included, which is accuracy scaled so pure guesses (at C=1) would give on average 0%.
• Moodle 2.1: Basic CBM interface behaviour was implemented in standard Moodle code, but CBM users are advised to upgrade from this version since proper CBM feedback is not available.
• Moodle 2.2-2.5: CBM gives marks 0.33, 0.67, or 1 for C=1,2,3 unless weights are all changed to 3. This is changed with code modifications so that all questions get marks 1,2,3 if correct. A 'No Idea' option is implemented with the patches. Feedback (with the modifications) includes both accuracy and CBM scores, with breakdown of performance by C level - as with Moodle 1.9-2.0.
• Moodle 2.6-2.7: The CBM interface and feedback to students were improved in core Moodle, and now work well. Default weighting of Qs gives marks 1,2,3 if correct and 0,-2,-6 if wrong, for C=1,2,3. Question weights ('max marks') can be varied to affect these marks in proportion, but this is not recommended to avoid confusion. NB CBM Grade Reports for teachers require installation of a PLUGIN, from the Moodle site, or from the download site here.

• Include a 'Don't Know' or 'No Idea' option, so students aren't forced to guess to receive feedback.
• Provide feedback (answers, explanations) only for Qs a student has attempted, not for all. Important for self-testing.
• Arrange the interface so submission of an answer can be done with one click (the chosen certainty), without need for a second 'enter' click.
• Show 'Knowledge' scores, scaled so that pure guesses would on average give a zero score.
• [Would improve all versions] Show the Moodle 'Grade' in gradebook summaries etc. as the CB Grade (reflecting CB Accuracy, up to 100%, which is the best comparison to make with conventional grades) rather than as the total of CB Marks (up to 300% of the maximum without CBM).

• In Moodle 2.6+, it is usually best to leave the weighting (or "maximum mark") for each question at its default value (=1) so that standard CBM scores (1,2,3, etc.) apply. These are straightforwardly interpreted by students and avoid confusion - for example whether a mark of 3 was for a confident answer at C=3, or an unconfident answer at C=1 on a question assigned triple weight.
• It can be confusing if, in a quiz with CBM, you include questions with multiple parts. The student may be sure about some parts and unsure about others. Individual parts are best asked separately with CBM, or else placed in a separate type of quiz that doesn't use CBM.
• At present, Moodle "Grades" - based on total marks - can be up to 300% with CBM. It is important to ensure that the option 'Unlimited Grades' is switched on under Site Administration -> Grades -> General settings. Otherwise, even very modest grades with CBM may show in some gradebook summary screens as 100%. There seems no evident situation in which it would be better to have this switch turned off.
• The CB Grade, shown in reviews of individual attempts and CBM Grade summaries, gives a proper comparison with conventional Grades (just based on accuracy). It is probably the best single number to use in student assessment, though not currently shown as the "Moodle Grade".