A.R. Gardner-Medwin
Dept. Physiology, University College London, London WC1E 6BT
It is important in physiology teaching that students know and understand the background and basic concepts in the subject. To identify and fill the diverse gaps in student knowledge, we must ask a lot of questions and provide immediate feedback and explanations. This is traditionally, but rather inefficiently, done in tutorials. Obviously it is an area where information technology should be able to help. However, automated assessment suffers from at least two problems considered here.
Firstly, it seldom makes use of information about how confident a student is in the answer given, which is part of what we take into account in assessing students person-to-person. Students must know for sure whether [Ca2+] is lower or higher inside cells, or whether 15 g is the same as 1500 mg. It is not good enough to have a hunch about the correct answer. The proposed scheme, which we have implemented at UCL, asks students, after each answer, to declare a confidence level of 1, 2, or 3. If the answer is correct, then this is the mark awarded. If not, marks of 0, -2, or -6 are awarded. Students do well on this scheme if they can discriminate between when they are sure of correct answers and when they are partly guessing. When doing exercises, students are trained to reflect on their reasoning and to develop an important study skill, which is to know whether you really understand something.
The second problem is that automated assessment often requires the construction of complex questions to ensure that students cannot get good marks through guesswork. Such questions can be ambiguous and open to different levels of interpretation, so the creation of satisfactory MCQ tests is time consuming. Confidence assessment helps to alleviate this problem, since it becomes less important to ask complex questions. Simple direct questions discriminate better between strong and weak students. Good students answer correctly with high confidence, while weak students moderate their confidence level if they know they are uncertain, or else lose heavily when they make mistakes. Data are presented based on over 100,000 questions asked in voluntary self-assessment amongst medical students.
The software and exercises (called LAPT: London agreed protocol for teaching) can be downloaded by Internet from ftp.ucl.ac.uk in the directory pub/users/cusplap or via World-Wide-Web from ftp://ftp.ucl.ac.uk/pub/users/cusplap.
Go to LAPT Home Page .
Queries / Problems: e-mail to cusplap@ucl.ac.uk