Predictive validity of continuous assessment score on the final examination score and assessment outcomes for pharmacy students examined between 2013 and 2017 at the University of Zambia

Abstract

Background: The University of Zambia, School of Medicine/ Health Sciences has an examination policy that puts significant emphasis on the contribution of the learners’ continuous assessment scores towards their final examination scores. While there is evidence of the predictive nature of continuous assessment on academic performance in some educational setups, there exists locally, insufficient evidence on the predictive validity of continuous assessment scores on the final examination scores and assessment outcomes of pass or fail and the grade point average in health professions education such as pharmacy. Realizing that the ultimate key determinants of course specific pass or fail and grade point average are the quality of the assessment practices and decisions, in the context of their validity, the current study proposes that the predictive validity of continuous assessment scores on the final examination scores and assessment outcomes of pass or fail and grade point average in promoting quality in health professions education be established. Therefore, this study set out to establish the predictive validity of the continuous assessment score on the final examination score and assessment outcomes of pass or fail and grade point average for pharmacy students examined between 2013 and 2017 while exploring the experiences and perceptions of examinees and examiners regarding the assessment policy and practice at the University of Zambia. Methods: The current study utilised a mixed-method approach using the convergent parallel study design. The quantitative arm was a non-interventional cross-sectional study, while the qualitative arm was a case study design. The study was conducted at the University of Zambia, School of Medicine, Department of Medical Education Development and Evelyn Hone College of Health Sciences located within Lusaka District, Zambia. The study sample comprised of the fourth- and fifth-year pharmacy students’ examination results at the University of Zambia and third-year pharmacy student’s examination results at Evelyn Hone College examined between 2013 and 2017. A total of 855 examination results were retrieved and analysed in the quantitative arm. In the qualitative arm, three (3) focus group discussions were conducted comprising of the third-year pharmacy students from Evelyn Hone College and a combined fourth- and fifth-year pharmacy students from the University of Zambia. Another focus group discussion was conduct on the examiners from both institutions. Correlations of the continous assessment score on the final examination score and the grade point average were carried out. Chi-square was used to establish the relationship between the continous assessment score and the course-specific pass or fail. Furthermore, Multiple linear and logistic regressions were conducted to establish the predictive validity of the continous assessment score on the final examination score, pass or fail and the grade point average while adjusting for demographic characteristics. In the focus group discussions, the data were transcribed and then compared and further regrouped and recoded to come up with broader themes based on the interview guide. The broader themes were then linked to the six core categories for the examinees focus group discussions while only three core categories were identified for the examiners focus group discussions. Results: There was a statistically significant positive correlation between course-specific continous assessment scores and the final examination scores as well as the grade point average scores across all the courses examined. The surprising finding after further analysis was that the median grade point average scores observed in the study were below the University of Zambia School of Medicine acceptable grade point average score of 2.5. There was a statistically significant difference observed between the course-specific mean continous viii assessment scores and the respective final examination scores. Multiple linear and logistic regression revealed that the continous assessment score had predictive validity on the final examination score, grade point average and pass or fail assessment outcomes. Additionally, examinees, whose continous assessment score was Less than 20 demonstrated capabilities of passing in all the courses examined, who in this context, would not have ordinarily been allowed to sit for the final examination . The main themes identified in the focus group discussions include: Awareness of the policy; Strengths and weaknesses; Feedback mechanism on assessments; and Factors affecting academic performance. Among the highlighted factors affecting students’ academic performance include: Poor communication among faculty; Inadequate time allocation for the study break; and High student to lecturer ratio resulting in poor individual student attention. Conclusion: There is predictive validity of the continous assessment score on the final examination score and examination outcomes of course-specific pass or fail and grade point average. However, it was observed that the median grade point average score obtained was lower compared to the University of Zambia acceptable grade point average score of 2.5. This could be indicative of poor mastery of expected competencies based on the definition of grade point average. Additionally, examinees whose continous assessment score was less than 20 demonstrated capabilities of passing in all the courses examined. The results suggest that there may be no justification to deny a student to sit the final examination based on the failed continous assessment score realising that a course is graded on a performance continuum of scores ranging from 0 to 100%. The current study has provided valuable frameworks and evidence for designing interventions to improve learning and performance in assessments leading to better pedagogical delivery of curriculum objectives and better decisions on the assessment of competencies. This will ultimately lead to better educational outcomes that will translate into competent health professionals to deliver quality health care services for the benefit of patients. Key Words: Predictive Validity, Continous Assessment, Final Examination, Summative Assessment, Grade Point Average