Research article
Open Access

Video-Assisted Self-Assessment of Basic Life Support skills in Medical Students

Ramya Ramakrishnan[1], Ramakrishnan Trichur Venkatakrishnan[1], Parthasarathy Vijayan[1][a], Srihari Cattamanchi[1][b]

Institution: 1. Sri Ramachandra Medical College and Research Institute, Porur, Chennai - 600116. India.
Corresponding Author: Dr Srihari Cattamanchi ([email protected])
Categories: Assessment, Educational Strategies, Teaching and Learning, Clinical Skills, Undergraduate/Graduate
Published Date: 30/05/2019

Abstract

Aim

This study seeks to evaluate the utility of video-assisted self-assessment in the attainment of basic life support (BLS) skills in medical students.

Methods

On procuring approval from the Institutional Ethics committee, third-year medical students were enrolled in the study. A large group interactive session on BLS skills was conducted followed by a demonstration of BLS skills on a manikin by the instructor. The students’ performance was recorded using an Apple iPad. Self-assessment was done by the students twice using a standard checklist – before and after watching the video recordings of their performance. The disparity in the self-assessment scores before and after watching their video was analyzed, which was compared with that of the instructors’ feedback, using students ‘t’ test and Mann Whitney test. Feedback about this assessment methodology was obtained using a questionnaire.

Results

Forty-one 3rd year medical students participated. A significant difference (<0.05) in the students pre and post video assessment scores was observed. All students agreed that watching the video recording of their skill performance helped them identify the errors better.

Conclusions

In conclusion, video-assisted assessment is an effective and a feasible method to improve self-assessment accuracy regarding clinical skills.

Keywords: Video-Assisted Self-Assessment; Self-Assessment; Basic Life Support; BLS Skills; Medical Students; MBBS Students

Introduction

Basic life support (BLS) is an essential life-saving skill for all healthcare staff, including medical students. Standardized educational programs and regular training (Vnuk, Owen and Plummer, 2006) are needed to maintain competence at all times to ensure adequate performance in crisis situations such as cardiac arrest. BLS skills are a set of clinical skills taught to undergraduate medical students' world over. The individual tutors and instructors assess these clinical skills. Self-assessment by the students helps them to identify deficiencies and correct them. Self-assessment of clinical skills is usually done using a standard checklist which is employed by the tutors. An added armamentarium is to video record the student’s skill performance. Video recordings of the simulation-based performance of clinical skills may provide the students with the possibility to analyse, appraise, and reflect upon their performance and offer a forum whereby practice-based learning can be educated, learned and evaluated.

Accurate self-assessment of performance (Eva and Regehr, 2005) enables students to recognise their strengths and weaknesses and identify which areas to concentrate on in their enduring medical education. Hence, self-assessment (Epstein, 2007) is recognised as a component of developing clinical competence in medical schools.

Methods

This study is a prospective cohort study of medical students - third year Bachelor of Medicine, Bachelor of Surgery (MBBS) degree conducted at a Level 1 Trauma Centre and a Tertiary Care University Teaching Hospital at Chennai in India. All the students had participated in an educational course, including lecture and demonstration followed by hands-on practice of BLS skills on a manikin during their first year of medical school.

The study was started after obtaining permission from the Institutional Ethics Committee. The third-year medical students rotating through the Emergency department were recruited for the study after taking their written informed consent. A large group interactive session with a video demonstration of BLS skills was conducted followed by a demonstration of BLS skills on a manikin by an AHA (American Heart Association) certified instructor. Students practised on the manikin (NASCO CPR PROMPT®, USA) on two occasions before the assessment.

Every student performed One person-rescue adult BLS in an unwitnessed cardiac arrest scenario using the manikin. The students' performance was recorded using an Apple iPad. Instructors and students made assessments during each simulation. The instructors assessed the students' performance before and after watching the video recordings using a standard checklist. The educational goals of the AHA Guidelines 2010 (International Liaison Committee on Resuscitation [ILCOR] Guidelines) formed the base of the checklist. The students used the same checklist for self-assessment. They then watched the video recordings and re-assessed themselves using the checklist. The difference in the self-assessment scores before and after watching their video was analyzed, which was compared with that of the instructors' assessment, using students' 't' test and Mann Whitney test. The students' feedback regarding this evaluation methodology was obtained using a questionnaire.

Data Analysis

Computations for continuous variables were done using means and standard deviation (SD). The intraclass correlation coefficient was employed to compare the performances scores between tutor and self-assessment. Statistical software SPSS (SPSS Version 22, SPSS Inc., Chicago, Illinois, USA) was utilised for data analysis. A p-value of <0.05 is observed as statistically significant.

Results/Analysis

A total of 41 third-year medical students partook in the study. Mean age was 20 years (range 20 to 22 years old). Twenty-five students were females, and 16 were male students. The students reduced their scores after watching their video recording which indicates that it helped them detect more mistakes than just scoring themselves from memory (Table 1). This difference in the scoring was statistically significant (p <0.05). The video recording will help the students realize their shortcomings and improve their skills in the future. The tutors scoring did not significantly differ before and after watching the video recordings of the student performance. This difference augurs well with the instructor's observation power (Table 2).

Table 1. Difference in Pre and Post Video scores

 

Mean

N

Std. Deviation

PAIR 1

Student Pre-Video

8.2195

41

1.50851

Student Post-Video

7.3171

41

1.23367

PAIR 2

Tutor Pre-Video

8.1220

41

1.77757

Tutor Post-Video

7.7073

41

1.41852

 

Table 2. The difference in the scores between students and tutors. Student's scores were significant (p<0.05)

 

 

Paired Difference

t

df

 

Sig. (2-Tailed)

Mean

Std. Deviation

95% Confidence Interval of the difference

Lower

Upper

PAIR 1

Student Pre-Video

0.90244

0.91665

0.61311

1.19177

6.304

40

0.000

Student Post Video

PAIR 2

Tutor Pre-Video

0.41463

1.39599

-0.02599

0.85526

1.902

40

0.064

Tutor Post-Video

 

When comparing the pre-video scores of the students' self-assessment vs the tutors' pre-video assessment, there was no significant difference. There was no difference similarly while comparing their post-video scores either (Table 3). The lack of difference indicates that the third-year medical students were as good at self-assessment of their BLS skills as were the tutors.

 

Table 3. Comparison between Student and Tutors pre and post video scores. Both the scores were not significant (p > 0.05)

 

 

 

t

df

Sig. (2-tailed)

Student Pre-Video

Vs

Tutor Pre-Video

Equal variances assumed

0.268

80

0.789

Student Post-Video

Vs

Tutor Post-Video

Equal variances assumed

-1.329

80

0.188

 

We found a correlation between student self-assessment scores and also tutor assessment scores. Intra-class Correlation (at 95% Confidence Interval) between student pre-video and post- video scores was 0.779 (range 0.622 to 0.876) and was significant. Intra-class Correlation (at 95% Confidence Interval) between Faculty pre and post video scores was 0.623 (range 0.393 to 0.780) and was significant (though the range is extensive). Analysis of the discrepancy among self-assessment and tutor assessment (Kim et al., 2011) will assist in identifying the actions overlooked by medical students and their demonstrated incompetence, on which the training can be focused.

Feedback from the students was analyzed. Ninety per cent of the medical students strongly agreed that the checklist was easy to use for self-assessment. All agreed that watching the video recording of their skill performance helped them identify the errors better. Only 29% of them felt nervous when their skill performance was being video-graphed. Sixty-eight per cent felt comfortable about the video recording. All agreed that the feedback from the instructor’s post-video was effective in their learning of BLS skills. Eighty per cent of the students felt strongly that this form of self-assessment could be used for learning other clinical skills.

Discussion

Assessment drives learning. Self-assessment in medical education helps develop medical students’ capabilities, knowledge, values, and skills, thus building a base to foster lifetime learning. Self-assessment  (Motycka et al., 2010) is defined as a process that empowers students to employ ascertained criteria to assess and reflect upon their learning to recognise domains of improvement and proficiency.

Andrade (Andrade and Du, 2007) explains self-assessment as a process of developmental evaluation throughout which students ponder on and appraise the character of their work and learning, assess the extent to which they reflect the declared objectives or benchmarks explicitly, recognise strengths or deficiencies in their performance and improve respectively. According to Weiss et al., (Weiss et al., 2005) self-assessment is a ‘‘process’’ intended to permit an individual to obtain information about his/her performance and analyse it with the objectives and/or the benchmarks for his/her work.

Studies (Sargeant et al., 2008; Eva and Regehr, 2011) have shown that medical students and junior doctors possess inadequate self-assessment capabilities and state that the determinants that impact informed self-assessment are complicated. In the particular setting of the self-assessment of practical skills (Evans, Leeson and Petrie, 2007), prevailing evidence proposes that healthcare trainees’ self-assessments are unreliable when compared with standard assessment of work.

Several researchers (Zick, Granieri and Makoul, 2007) have assessed the effectiveness of video analysis in the self-assessment of clinical and/or communication skills. Moreover, the video-recording of student work in the healthcare fields has been appraised as a tool to enhance the quality of self-assessment. Video-recording has been employed for educational purposes in numerous healthcare professions including physical therapy, nursing, medicine, and surgery with mixed outcomes.

In a study by Ward et al., (Ward et al., 2003) laparoscopic Nissen fundoplication performed in a pig by 26 senior surgical residents were video recorded. Using two scoring systems, the experts rated the videos. The surgical residents assessed their performance immediately following the laparoscopic Nissen fundoplication, after self-observation of the video of their performance, and eventually after reviewing videos of the four "benchmark" performances. The inter-rater reliability of the experts was 0.66 (intra-class correlation coefficient). Initial comparison of the experts' and the surgical residents' self-evaluations showed a moderate correlation (r = 0.50, P <0.01). The correlation significantly increased after the surgical residents reviewed the video of their own performance to r = 0.63 (Delta r = 0.13, P <0.01). However, after a review of the benchmarks, there was no change in the correlation. Reviewing their own performance on video improved the surgical residents' ability to self-assess. Similar results were observed in our study.

Video-assisted self-assessment has been used in the learning of Basic life support skills too. Vnuk et al., (Vnuk, Owen and Plummer, 2006) used video review in his study of 95 students in BLS training. The student's evaluations did not correlate with the expert assessments either before or after video analysis.

Sadosty et al., (Sadosty et al., 2011) used videotaped simulations in her study of 17 postgraduate emergency medicine (EM) residents. The EM residents reviewed their skills in a case before and after watching the video of their own performance. These evaluations were then correlated to faculty assessment reports. The video analysis did not increase the correlation of the EM resident scores to those of the faculty evaluators.

Ozcakar et al., (Ozcakar et al., 2009) mentioned one of the limitations of his research on history taking and communication skills was that the evaluators were not blinded to the participants. This limitation can affect the validity of the results, even if steps were taken to reduce the potential bias with a well-defined and stringent rating criterion.

Video-recording presented a unique opportunity for the third-year medical students to witness his/her performance, and its advantage was additionally amplified by the comments of the instructors as seen in our study. Although video-recording performs an essential role in learning, it may additionally create some difficulties. One of these deficiencies is the distress perceived by medical students. A few researchers (Nilsen and Baerheim, 2005) observed that students perceive anxiousness and oppose video recordings. Some studies (Mar and Isaacs, 1992) observed that students needed to perform more than one video-recording. In our study, only 29% of the students felt nervous about videotaping their performance. The majority felt quite comfortable about recording their performance.

One of the limitations of our study was that the students were not formally taught in the technique of self-assessment. This lack of self-assessment skills could have led to the students over-rating themselves before watching their performance on the video-recording. Though a significant disparity was observed in the student self-assessment scores before watching the video and after, this should be tested on a larger sample of participants ideally.

Conclusion

Continuous self-assessment of knowledge and skills is quintessential for physicians. Self-assessment competency at the start of postgraduate training is undoubtedly connected with the active involvement in targeted procedure course during medical school and the numerous procedures done as a medical student. Formal training in self-assessment technique should start during a student's medical school (undergraduate medical education), with the expectation that complete proficiency in self-assessment is accomplished during postgraduate medical training and maintained after graduation through continuing medical education.

This study highlights issues which may significantly influence the use of video playback in self-assessment. It has identified a potentially efficient and feasible method for developing self-assessment accuracy regarding clinical skills. Additional study is required for more number of participants and testing a broad range of clinical skills with increasing complexity before we can pronounce that this method of self-assessment is useful for medical students.

Take Home Messages

Self-assessment is a vital instrument for lifelong medical education. A video-based self-assessment method is a valid tool in skill training of undergraduate medical students. Simulation-based performance of clinical skills contributes to practice-based learning.

 

Notes On Contributors

 

Prof. Dr Ramya Ramakrishnan, MBBS, MS (General Surgery), FRCS (Ed.), has 30 years of clinical experience and specialises in General Surgery, Breast and Endocrine Surgery. She is a Professor in the Department of General Surgery, Secretary of the Medical Education Unit (MEU), PhD Guide and an Examiner for undergraduate and postgraduate students at Sri Ramachandra Institute of Higher Education and Research, Chennai, India. ORCID ID: https://orcid.org/0000-0003-3634-1184

Prof. Dr Ramakrishnan Trichur Venkatakrishnan, MBBS, MD (Anaesthesiology), is the senior consultant and head of the emergency department with more than 20 years of clinical experience in Emergency Medicine at Sri Ramachandra Medical College and Research Institute, Chennai, India. ORCID ID: https://orcid.org/0000-0001-9545-6294

Mr Parthasarathy Vijayan, BSc (Emergency & Trauma Care Technologist), Postgraduate Diploma in EMS, MSc (Trauma Care Management), with 12 years of experience as a paramedic, is currently working as a lecturer for paramedic students at the Division of Faculty of Allied Health Sciences at Sri Ramachandra Institute of Higher Education and Research, Chennai, India. ORCID ID: https://orcid.org/0000-0003-1699-7311

Dr Srihari Cattamanchi, MBBS, MD (Accident & Emergency Medicine), MPH, is an Adjunct Professor (Research) in the Department of Emergency Medicine at Sri Ramachandra Medical College and Research Institute, Chennai, India. Dr Srihari is the Director of Humanitarian Disaster Operations at BIDMC - Fellowship in Disaster Medicine, Boston, USA. ORCID ID: https://orcid.org/0000-0002-8540-1115

Acknowledgements

None.

Bibliography/References

Andrade, H. and Du, Y. (2007) ‘Student responses to criteria‐referenced self‐assessment’, Assessment & Evaluation in Higher Education. Routledge, 32(2), pp. 159–181. https://doi.org/10.1080/02602930600801928

Epstein, R. M. (2007) ‘Assessment in Medical Education’, New England Journal of Medicine, 356(4), pp. 387–396. https://doi.org/10.1056/NEJMra054784

Eva, K. W. and Regehr, G. (2005) ‘Self-assessment in the health professions’, Academic Medicine, 80(10), pp. S46-S54. https://doi.org/10.1097/00001888-200510001-00015

Eva, K. W. and Regehr, G. (2011) ‘Exploring the divergence between self-assessment and self-monitoring’, Advances in Health Sciences Education, 16(3), pp. 311–329. https://doi.org/10.1007/s10459-010-9263-2

Evans, A. W., Leeson, R. M. A. and Petrie, A. (2007) ‘Reliability of peer and self-assessment scores compared with trainers’ scores following third molar surgery’, Medical Education, 41(9), pp. 866–872. https://doi.org/10.1111/j.1365-2923.2007.02819.x

Kim, S.-J. et al. (2011) ‘The analysis of self and tutor assessment in the skill of basic life support (BLS) and endotracheal intubation: Focused on the discrepancy in assessment’, Resuscitation. Elsevier, 82(6), pp. 743–748. https://doi.org/10.1016/j.resuscitation.2011.01.031

Mar, C. Del and Isaacs, G. (1992) ‘Teaching consultation skills by videotaping interviews: A study of student opinion’, Medical Teacher. Taylor & Francis, 14(1), pp. 53–58. https://doi.org/10.3109/01421599209044016

Motycka, C. A. et al. (2010) ‘Self-Assessment in Pharmacy and Health Science Education and Professional Practice’, American Journal of Pharmaceutical Education. American Association of Colleges of Pharmacy, 74(5), p. 85. https://doi.org/10.5688/aj740585

Nilsen, S. and Baerheim, A. (2005) ‘Feedback on video recorded consultations in medical teaching: why students loathe and love it -- a focus-group based qualitative study’, BMC Medical Education, 5(1), p. 28. https://doi.org/10.1186/1472-6920-5-28

Ozcakar, N. et al. (2009) ‘Is the use of videotape recording superior to verbal feedback alone in the teaching of clinical skills?’, BMC Public Health, 9(1), p. 474. https://doi.org/10.1186/1471-2458-9-474

Sadosty, A. T. et al. (2011) ‘Simulation-based Emergency Medicine Resident Self-Assessment’, Journal of Emergency Medicine. Elsevier, 41(6), pp. 679–685. https://doi.org/10.1016/j.jemermed.2011.05.041

Sargeant, J. et al. (2008) ‘“Directed” self-assessment: Practice and feedback within a social context’, Journal of Continuing Education in the Health Professions, 28(1), pp. 47–54. https://doi.org/10.1002/chp.155

Vnuk, A., Owen, H. and Plummer, J. (2006) ‘Assessing proficiency in adult basic life support: student and expert assessment and the impact of video recording’, Medical Teacher. Taylor & Francis, 28(5), pp. 429–434. https://doi.org/10.1080/01421590600625205

Ward, M. et al. (2003) ‘Resident self-assessment of operative performance’, The American Journal of Surgery. Elsevier, 185(6), pp. 521–524. https://doi.org/10.1016/S0002-9610(03)00069-2

Weiss, P. M. et al. (2005) ‘How do medical student self-assessments compare with their final clerkship grades?’, Medical Teacher. Taylor & Francis, 27(5), pp. 445–449. https://doi.org/10.1080/01421590500046999

Zick, A., Granieri, M. and Makoul, G. (2007) ‘First-year medical students’ assessment of their own communication skills: A video-based, open-ended approach’, Patient Education and Counseling. Elsevier, 68(2), pp. 161–166. https://doi.org/10.1016/J.PEC.2007.05.01

Appendices

None. 

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

Obtained Institutional Ethics Committee approval on 17th November 2014, Reference No: IEC-NI/14/OCT/43/70 from Sri Ramachandra University - Institutional Ethics Committee, Porur, Chennai - 600116. India.

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Ken Masters - (21/08/2019) Panel Member Icon
/
A small, interesting study dealing with video-assisted self-assessment of basic life support skills in medical students

Some issues needing attention:
• The Introduction is very short and thin. Two areas in particular need attention:
o The references: several statements on the assessments are made, but almost none of them are supported by literature. The authors may wish to draw on some of the material that they use in the Discussion
o No institutional background or aim is stated. There is an aim in the abstract, and this could be used in the body of the paper’s Introduction, but that abstract’s introduction also needs to be expanded.

• When numbers are reported in the text, the percentages should also be given (e.g. “Twenty-five (60.98%) students…” At other times, only percentages are reported, and, again, this should be done with both raw numbers and percentages.
• The p values should not be given merely as p<0.05. (This applies to all p values in the paper). In Table 1, the actual p values of both the students and tutors should be given.

• “The video recording will help the students realize their shortcomings and improve their skills in the future.” This sentence is in the Results, and should rather be part of the Discussion.

• The Discussion should rather be a Discussion of the Results in light of the literature. In this paper, however, the Discussion is much more of a literature review, most of which is better suited to the Introduction, especially given that the Introduction is so thin on literature.


Small issue:
o Table 1: Given that n remains 41 throughout, the n=41 can be stated in the table’s heading rather than repeated in the column.
o Some small language issues (e.g. “The tutors scoring”) and elsewhere, need to be corrected.

So, an interesting small study, but the paper does need some attention.

Possible Conflict of Interest:

For Transparency: I am an Associate Editor of MedEdPublish

Debra Kiegaldie - (30/06/2019) Panel Member Icon
/
This is a small cohort study to examine the impact of video assisted self-assessment for the learning of BLS skills. The intervention has the potential to be useful for this type of skill acquisition. It would be worth repeating this study with a larger cohort and I would like to see the addition of how this skills is retained over time e.g. 1 month post, 6 months post etc. Some small rephrasing of sections would also be helpful. E.g. ethics approval process. The opening sentence in the discussion is unnecessary – I would just focus on self-assessment as a process of learning. I also think some of the literature findings in the discussion need to be bought forward in the introduction or background of paper.
priya Prabhakaran - (09/06/2019)
/
This study highlights an effective use of inexpensive video technology to assist learners in the process of real time self assessment which will potentially translate into better performance . The numbers enrolled are relatively small and it would be worth repeating this study with more subjects.
Possible Conflict of Interest:

None

Felix Silwimba - (30/05/2019)
/
The study provides an example of use of low cost affordable modern learning aids. It strengthens student centered learning and self-assessment. I agree with the authors it will build into lifelong learning in medical doctors. It as well addresses the observation that students today are visual learners.