A.R. Gardner-Medwin, Dept. Physiology, UCL, London WC1E 6BT
UPDATING with CONFIDENCE
Do your students know what they don't know ?
An innovation at University College London is the use, over the last 3 years, of confidence assesment in students' computer-based study. This has been developed by Dr. Tony Gardner-Medwin (Reader in Neurobiology) in a system called LAPT (London Agreed Protocol for Teaching). It is designed to encourage the habit of thinking about how sure you are of the basis of your knowledge, every time you answer a question. The initiative began as a collaboration between the physiology departments in several London medical schools. Since then it has expanded its scope and is being used for anatomy and pharmacology and for practice in numeracy skills as well as in physiology.
In tutorial situations we judge students in two ways:- firstly,
by the correctness of their answers, and secondly by the confidence with
which they give these answers. Confidence may be indicated by the tone
of voice or body language, or just by the latency of the answer. Confident
right answers are generally a sign of sound knowledge, while confident
wrong answers are indicative of a much more serious problem than are errors
through acknowledged guesswork. Especially in medicine, we must be wary
of students who do not know when they might be wrong.
LAPT makes this distinction by asking students to grade their confidence at one of 3 levels, 1, 2 or 3. If the answer turns out to be right, this is the number of marks awarded (1, 2, or 3). If the answer is wrong, then at the lowest level of confidence there is no penalty while at higher levels there is progressively more severe negative marking (-2, -6 respectively). Analysed in terms of game theory, students are best to opt for level 2 when 67% sure of their answer, and level 3 when more than 80% sure. The need to think about confidence encourages students to check their answers, re-read the question, explain to themselves why their answer is right and relate it to other pieces of knowledge. Forty percent of students in a recent evaluation survey said they sometimes change their answer when thinking about confidence. All of this helps to build up a network of cross-referenced knowledge, the sort of thing we call 'understanding' as opposed to the rote learning of individual facts. Students respond constructively to the jolt when they get things wrong despite being confident, or when they dare not bet on even the basics of the subject.
LAPT is currently used mainly in voluntary self-directed study
by preclinical medical students. Usage has been escalating (see Figure).
About 2/3 of the students in the evaluation survey claim to find the confidence
element "Useful" or "Very Useful". They readily understand the importance
of distinguishing confident knowledge and guesswork. After the mark is
awarded, explanations of the topic are available for a large fraction of
the questions under LAPT. Students find this a useful adjunct to bookwork,
often reinforcing knowledge from different angles. LAPT includes many varied
question and answer formats, but even for simple True/False questions the
confidence assessment and attention to student comments about individual
questions are helping to counter the poor image that computer-marked exercises
often engender.
A notable change in the usage pattern in 1996/97 was the more uniform
use throughout the year, not just concentrated at revision times before
exams. This results from deliberate efforts to incorporate computer work
as part of the students’ basic study regime. LAPT runs on PCs under Windows.
Students can easily take discs home. Our survey in 1996 suggested that
the usage recorded on campus computers (shown in the Figure) accounts for
only about 40% of total LAPT use. About 60% is home study. This surprising
statistic is important for a policy of development. We want to encourage
home use as much as possible, because this vastly increases the usable
hardware resources for teaching, beyond what the College can provide.
Students prefer to study at home. But there is a potential downside to home use, which we believe we can minimise through recent developments. Home installations of LAPT can rapidly become out-of-date, lacking the corrections and additions that are regularly made to UCL files. The conventional approach to this problem is to convert everything to the World-Wide Web, using browsers to access information. Students are increasingly acquiring modems on home computers. But even with a modem and a bulk phone discount, nobody wants to sit studying on a phone-line all evening when some long-awaited romantic phone call might unhappily encounter the British Telecom engaged tone. The LAPT solution is an arrangement whereby students get LAPT to initiate automatic update by FTP (File-Transfer-Protocol), perhaps once a fortnight or so. With a brief call the server is interrogated to establish which parts of the student's installation require updating. The student can then choose those parts of the system that he or she wants to install or bring up-to-date. All this is complete in a few minutes. Study work is all done directly on the local computer, so feedback is instantaneous after answers and confidence have been entered, with no irritating network delays.
UCL encourages other institutions and individuals to experiment with the system. There is an institutional registration fee of £100 for continued use after a month, and one of the principal conditions is that institutions share any material developed or adapted for LAPT. One of the strengths of LAPT is the ease with which material can be tailored and presented in different ways for local course structures. It can also introduce, explain and launch other existing teaching software. Load up your browser right now and go to http://www.ucl.ac.uk/~cusplap, from where the manuals, information about publications and the system can be downloaded.
For more information contact Dr. A.R. Gardner-Medwin, Dept. Physiology, UCL, London WC1E 6BT
(E-mail a.gardner-medwin@ucl.ac.uk)or visit the web site http://www.ucl.ac.uk/~cusplap