ISSN-0303-5212
  The Journal of the Pakistan Medical Association
Rawalpindi - Islamabad		  
     
Volume 34, Number 2, July - Dec. 2009
     
 
MEDICAL EDUCATION
 
Designing the OSCE method for evaluation of practical immunology
course of medical students: in comparison to written-MCQ and
oral examination
 
Abdollah Jafarzadeh
 
Objective

To design an objective structured clinical examination (OSCE) method for evaluation of medical students in practical immunology course and to compare their perceptions regarding OSCE with Multiple Choice Questions (MCQ) and Oral examination.

Method

A total of 42 students who had practical immunology course were included in the study. A series of 10 stations were established for OSCE. At each station the students were given a patient description were asked to answer questions for results and interpretation of tests within two minutes. Then, the MCQ examination consisting of 10 questions and oral examination with 10 same questions were conducted. Finally, a questionnaire regarding the quality of examination, being easy, being educative, being fair, and being suitable for evaluating tools for skills and anxiety were filled by students.

Results

OSCE, MCQ and Oral examination were perceived easy by 66.7%, 81% and 81% of students and were considered educative by 90.5%, 59.5% and 76.2% of students, respectively. Educative value of OSCE was thought to be significantly higher than MCQ (P<0.003). OSCE, MCQ and Oral examination were fair by 85.7%, 90.5% and 73.8% of students, respectively. A large proportion of students believed the fairness of MCQ was significantly higher as compared to Oral examination (P<0.04). Higher proportion of students reported that OSCE were suitable tools for evaluation of the practical skills as compared to MCQ and Oral examination (78.6% v 54.8% and 52.4%, respectively; P<0.02). OSCE, MCQ and Oral examination were stressful for 64.3%, 23.8% and 73.8% of students. Anxiety of MCQ was significantly lower in comparison to OSCE and oral examination (P<0.00001). In all methods, the anxiety was higher in females in comparison to males but only in QSCE method the frequency of female who had anxiety (80%) was significantly higher as compared to males (50%, P<0.05). Moreover, 81% of students agreed for evaluation of practical immunology with using of OSCE method.

Conclusions

Although, higher proportion of students had stressful feeling during OSCE method, majority believed that the OSCE was educational, fair and suitable tools for evaluation of skills. The OSCE was an acceptable method by majority of students to assess practical Immunology skills. Accordingly, the OSCE should be considered for evaluation of practical courses of medical students. (Rawal Med J 2009;34:219-222).
 
Key Words: Evaluation, OSCE, Multiple choice questions, Oral examamination.
 
INTRODUCTION
 
Evaluation of the students learning is one of the essential functions of the academic centers. Effective evaluation not only increases the students' motivation but also helps instructors to determine the strenght or weakness of their educational activities for improvement of their performance.1,2 Many current methods of evaluation may be not suitable for assessments of clinical and practical competence of students.3,4 Bloom's taxonomy for levels of understanding defined six levels: knowledge, comprehension, application, analysis, synthesis and evaluation.5 Generally, written examination is best suited for evaluating level 1 to 3 and have little validity for higher levels.5,6

The objective structured clinical examination (OSCE) is a verified valid and reliable, constructive and summative tool for assessing the clinical skills learned by medical students.7 OSCE can assess students' clinical abilities in a comprehensive and standardized fashion.8 The main objective of OSCE is to evaluate students skills and attitudes at a higher level than is possible with traditional evaluation approaches such as written and Oral examinations.7,8 The way students were evaluated is a key factor in what they learn.9 In practical immunology course, in addition to knowledge, students skills about tests such as procedure, principal, reading of results and interpretation need to be evaluated. The aim of this study was to design an OSCE method for evaluation of medical students in practical immunology course and also to compare the students' perceptions regarding OSCE with Multiple Choice Questions (MCQ) and Oral exam.
 
SUBJECTS AND METHODS
 
A total of 42 medical students (22 males, 20 females; mean age: 20.93±1.43 year ) who had practical immunology course were included to study. A series of 10 stations were established for OSCE in the Immunology laboratory (Table 1). At each station, students were given a patient description with a performed practical test and were asked to answer questions on material provided, procedure, principals, reading the results and interpretation of tests within 2 minute. Then, the MCQ exam consisting of 10 questions was performed and an Oral exam with 10 same questions was conducted. Finally, a questionnaire regarding the quality of exam, the difficulty, being educative, being stressful, fair and being suitable was filled by students.
 
Table 1. Stations of OSCE for evaluation of practical Immunology.
 
Differences in variables were analyzed using Chi-square test and P values of less than 0.05 were considered significant. Data were analyzed by a computer program (SPSS, Chicago, IL, USA).
 
RESULTS
 
All students completed the questionnaire. Of the students, 22 were male with mean age of 21.13±1.83 year and 20 were female with mean age of 20.7±0.8 year. The points of view of students about OSCE, MCQ and Oral examination have been summarized in Table 2. Although, the frequency of students who believed that the OSCE was easy was lower than those observed for MCQ and Oral examination but the differences were not statistically significant. OSCE was significantly educative than MCQ (P<0.003).
 
Table 2. The points of view of students about OSCE, MCQ and Oral examination.

* represents the difference between OSCE and Oral exam.
** represents the difference between OSCE and MCQ.
*** represents the difference between Oral exam and MCQ
 
No significant differences were observed between the points of view of males and females with respects to the difficulty, being educative, being fair and being suitable evaluating tools of exams. Anxiety of MCQ was significantly lower in comparison to QSCE and oral examination (P<0.00001). In all methods the anxiety was higher in females in comparison to males. Moreover, 80.9% of students stated that the time was enough in each station and 81% of students agreed for evaluation of practical immunology with using of OSCE method.
 
DISCUSSION
 
OSCE is a reliable and valid scale for assessment of educational activities.7,8 The results of this study showed that the OSCE has been considered as an acceptable method to assessment of practical clinical skills of medical students. Moreover, they had positive attitudes towards OSCE as an alternative method to evaluate the skills in practical immunology course. The OSCE has been also accepted by medical students for evaluation of other courses.9,10 In this study, higher proportion of students believed that OSCE was easier in comparison to MCQ and Oral examination. The positive attitudes of students towards OSCE may not have been due to ease of OSCE but probably due to higher score in OSCE

Over 63% of students were stressful during OSCE evaluation, as reported in other studies.10 The reasons for stress may include inadequate prior instructions, newness of the pattern to students and their lack of experience with OSCE. In all examination methods the anxiety was higher in females in comparison to males but only in QSCE method the frequency of female who has anxiety was significantly higher as compared to males. The differences have been attributed to differential characteristics of psychoneuroendocrine mechanisms in males and females.11 The reasons for lower anxiety of MCQ may be due to prior instructions and prior experiences. Other studies in Nursing and Radiology students have demonstrated anxiety of OSCE significantly higher in comparison to written-MCQ.12-14

Our study also showed that the OSCE was fair as seen by majority (85.7%) of students. Students had no face to face encounter with examiner during OSCE which helped the students. Although, we have provided same conditions in Oral examination, lower proportion of students have feeling of fairness during Oral exam. We also showed that the higher proportions of students believed that the OSCE was educative and suitable tools for evaluation of the practical skills in comparison to MCQ and Oral examination, which has been reported by others.15 Thus, OSCE not only is a valid and reliable tool for assessment of practical skills but also encourages students to improvements their educational defects.
 
CONCLUSION
 
The results of our study showed that higher proportion of medical students has anxiety during OSCE method. However, the majority of students believed that the OSCE was educational, fair and suitable tools for evaluation. Thus, designing and incorporation of OSCE should be considered for evaluation of practical courses of medical students.
 
From Education affairs and Department of Immunology, Rafsanjan University of Medical Sciences, Rafsanjan, Iran.
Correspondence: Abdollah Jafarzadeh, Associate Professor of Immunology. Tel: +98 391 523 4003.
E-mail: Jafarzadeh14@yahoo.com
Received: September 1, 2009 Accepted: October 24, 2009
 
REFERENCES
 
  1. Baral N, Paudel BH, Das BK, Aryal M, Das BP, Jha N, et al. An evaluation of training of teachers in medical education in four medical schools of Nepal. Nepal Med Coll J 2007;9:157-61.
  2. Newble Dl, Jaeger K. The effects of assessments and examinations on the learning of medical students. Med Educ 1983;17:165-71.
  3. Newble DI. Assessing clinical competence at the undergraduate level. Med Educ 1992;26:504-11.
  4. Love RR, Newcomb PA, Schiller JH, Wilding G, Stone HL. A comparison of knowledge and communication skill evaluations by written essay and oral examinations in preclinical medical students. J Cancer Educ 1993;8:123-8.
  5. Bloom BS. Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain, Addison-Wesley, 1984.
  6. Gharibyan H. Assessing students' knowledge: oral exams vs. written tests. ACM SIGCSE Bulletin. 2005;37:143-47.
  7. Cohen R, Reznick RK, Taylor BR, Provan J, Rothman A. Reliability and validity of the objective structured clinical examination in assessing surgical residents. Am J Surg 1990;160:302-5.
  8. Iqbal M, Khizar B, Zaidi Z. Revising an Objective Structured Clinical Examination in a Resource-limited Pakistani Medical School. Education for Health, Volume 22, issue 1, 2009; Available from: http://www.educationforhealth.net.
  9. Dadgar SR, Saleh A, Bahador H, Baradaran HR. OSCE as a tool for evaluation of practical semiology in comparison to MCQ & oral examination. J Pak Med Assoc 2008;58:506-7.
  10. Chehrzad M, Shafiei pour Z, Mirzaei M, Kazemnejad E. Comparison between Two Methods: Objective Structured Clinical Evaluation (OSCE) and Traditional on nursing Students' Satisfaction. J Guilan Uni Med Sci 2004;13:8-13.
  11. Kosmala JA, Louis MW. Gender differences in the psychosomatic reactions of students subjected to examination stress. J Res Educ Psychol 2007;5:325-48.
  12. Brosnan M, Evans W, Brosnan E, Brown G. Implementing objective structured clinical skills evaluation (OSCE) in nurse registration programmes in a centre in Ireland: a utilisation focused evaluation. Nurse Educ Today 2006:26:115-22.
  13. Furlong E, Fox P, Lavin M, Collins R. Oncology nursing students' views of a modified OSCE. Eur J Oncol Nurs 2005:9:351-59.
  14. Marshall G, Jones N. A pilot study into anxiety induced by various assessment methods. Radiography 2003:9:185-91.
  15. Kowlowitz V, Hoole AJ, Sloane PD. Implementation of the objective structured clinical examination in a traditional medical school. Acad Med 1991;66:345-7.
pan
Abstract		 read
Full Article		 pdf
Download
 
All right reserved Rawal Medical Journal 2004 - 2008