Video Review in Self-Assessment of Pharmacy Students ’ Communication Skills

The objectives of this study were to develop a student self-assessment activity of a video-recorded counseling session and evaluate its impact on student self-perceptions of specific communication skills. This activity was incorporated into a core-communications course within the third professional year of a Doctor of Pharmacy curriculum. Student counseling sessions were video-recorded and released to students for self-assessment review. After watching their recorded counseling sessions, students completed an eight-question, paper-based survey which evaluated the impact of self-assessment on communication skill perceptions. Most students (95.6%) agreed or strongly agreed that their self-assessment process was valuable in developing their communication skills. The greatest change in assessment was associated with the use of eye contact (49.5%). Approximately 40% of students noted changes in perceptions for appropriate rate of speech (42.1%), voice volume (39.8%), and facial expressions (37.2%). Overall, self-assessment of video-recorded counseling sessions impacted students’ perceptions of both verbal and non-verbal communication skills.


Background
Self-assessment in professional education helps improve students' knowledge, abilities, and values, and builds a foundation to support lifelong learning.As defined, self-assessment is a process that allows students to utilize established criteria to evaluate and reflect upon their learning to identify areas of improvement and competence (Motycka, Rose, Ried, & Brazeau, 2010).This competency is supported by the Accreditation Council of Pharmaceutical Education [ACPE] (2006) with recommendations to include activities that support self-assessment into the pharmacy curriculum.Stated under Guideline 11.1 regarding teaching methods, the standards call for the program to encourage and assist students to assume "responsibility for their own learning (including assessment of their learning needs, development of personal learning plans, self-assessment of the level of their knowledge skills, attitudes, and values, and their achievement of desired competencies and outcomes)" (ACPE, 2006).This responsibility also has been noted to be critical for their success after graduation, with self-appraisal being a component of continual professional development in pharmacy continuing education (Fjortoft, 2006).

Review of Literature
Many healthcare professions rely on self-assessment as a means of continuing professional development.Established research has demonstrated the limitations, however, of students in professional education to accurately self-assess their competence.In a systematic review of 20 physician self-assessments versus external observations, 13 showed little, no, or inverse associations between the two.These assessments included evaluation of clinical skills, teaching abilities, cultural competencies, and understanding of evidence based medicine (Davis et al., 2006).A study of self-assessment of communication skills in over 400 medical students found that student values differed significantly from the standardized patients for the majority of communication related skills (Gruppen et al., 1997).In a similar review of 80 senior students in a Doctor of Pharmacy program, students, on average, overestimated their communication skills and clinical knowledge, as compared to external raters (peers and expert) (Austin & Gregory, 2007).
Various studies have evaluated the use of video review in the self-assessment of clinical and/or communication skills (Del Mar & Isaacs, 1992;Hays, 1990;Hu, Tiemann, & Brunt, 2013;Nilsen & Baerheim, 2005;Ozcakar et al., 2009;Zick, Granieri, & Makoul, 2007).In addition, videotaping of student performance in the healthcare professions has been evaluated as a tool to improve the quality of self-assessments.This has been utilized for education in various healthcare professions including nursing, physical therapy, medicine, and surgery with varied results (Lane & Gottlieb, 2004;Palmer, Henry, & Rohe, 1985;Yoo, Son, Kim, & Park, 2009).In a study of a videotaped simulation by emergency medicine residents (n=17), residents rated their skills on a case before and after watching their videotaped performance.These assessments were then compared to those of a faculty evaluator.Video review did not improve the correlation of the resident scores to those of the faculty member (Sadosty et al., 2011).In a similar study of the use of video review for students (n=95) in basic life support training, the student assessments did not correlate with the expert evaluation either before or after video review (Vnuk, Owen, & Plummer, 2006).In a quantitative and qualitative review of the use of videotaping in simulated resuscitations and the impact of self-assessment for pediatric residents, feedback from study participants noted that the video review was beneficial, although challenging.Differences between observer assessments and self-assessments did not vary widely, however, the residents described less usefulness of observer assessments.After video review, the majority of residents lowered their scores, and the residents described this activity as the most useful aspect of the study.In a nursing course, self-assessment by video review of vital signs and communication skills lead to significantly better performance on an exam as compared to self-assessment by written description (Yoo et al., 2009).
The impact of videotape review on pharmacy students' self-assessments of communication skills was recently studied by Mort and Hansen (2010).First-year students in a pharmacy practice course participated in a digitally recorded counseling exercise.Correlation between students' scores and faculty's scores, pre-and post-videotape review were compared and found to be poor.Videotape review, however, was found to change students' own rating of their counseling session.In the areas of technical information (details regarding indication, adverse effects, etc.) and interpersonal skills, the students' pre-and post-video scores were significantly different (p=0.001 and p<0.0001, respectively) than those prior to video review.Although the correlation to faculty ratings did not improve in this case, activities such as these may provide students the opportunity to practice and utilize the videotape as a tool to assist in the recognition of strengths and weaknesses (Mort & Hansen, 2010).
Communication skills are an essential area for patient centered practice, as outlined in the principles of pharmaceutical care by the American Pharmacists Association [APhA] (1995).This skill is also supported by ACPE, as well as the recently revised Center for Advancement of Pharmacy Education (CAPE) outcomes (Medina et al., 2013).CAPE guidance suggests that entry level graduates should be able to communicate effectively both verbally and nonverbally, during interactions with patients, as well as with other individuals or groups.Areas for focus include technical information in addition to verbal and nonverbal skills, such as body position, eye contact, and gestures (Beardsley, Kimberlin, & Tindall, 2008).

Course Description
At The Ernest Mario School of Pharmacy, the Pharmacy Communications course series is comprised of two separate two-credit courses delivered over the fall and spring semesters of the third professional (P3) year of the Doctor of Pharmacy curriculum.Both courses utilize six small class size sections (approximately 30-35 students) and integrate lectures, group activities, and one-on-one counseling sessions for the development of student interpersonal, professional, and clinical communication skills.The weekly two hour course is coordinated by three full time faculty members with each section led and taught by two adjunct faculty members from post-graduate programs.Course materials, activities, and grading evaluations are developed by the faculty coordinators, with delivery of lecture materials and individual section direction by the adjunct faculty.Approximately one-half of the classes are comprised of one-on-one counseling sessions.Topics for counseling focus on a wide variety of prescription and nonprescription medications.
Course topics are generally introduced via lectures delivered by the adjunct faculty, followed by an interactive breakout session to practice skills and/or apply material taught.Examples of topics discussed and evaluated include introduction to counseling, professionalism, how to give and receive feedback, presentation skills, journal clubs, and documentation of pharmaceutical care services.This course is well suited for the inclusion of a self-assessment activity due to small class sizes and inclusion of various individualized activities.
Incorporation of self-assessment in the pharmacy curriculum is critical for students in order to utilize this skill upon graduation.Research has demonstrated varied results regarding the benefit of video review on self-assessment, with limited information in pharmacy education.In this study, the implementation of a video-recorded prescription medication counseling session and its impact on self-assessment of specific communication skills for students in the P3 year are described.

General Counseling Instruction
This study was conducted over a three-week period of the Pharmacy Communications course involving one lecture/counseling practice day, one counseling practical day and one feedback/grade review day.The first class day introduced patient counseling through a small group lecture with sample counseling videos followed by an interactive breakout practice session.The introduction to counseling lecture provided a thorough review of the Indian Health Services (IHS) style counseling method which gauges a patient's understanding of their medication(s) through the use of three open-ended questions regarding purpose, dosage/administration, and outcomes (Gardner, Boyce, & Herrier, 1991).Using the open-ended questions to guide the counseling session, the pharmacist supplements information provided by the patient and verifies that the patient understands the information.Verification occurs at the conclusion by having the patient describe how they will take the medication.Any discrepancies, missing information, or incorrect information can be addressed at the conclusion of the session (Gardner et al., 1991).
The IHS counseling method review established the expected counseling style and structure for students to follow when counseling mock patients during practical assessments.Based on this method, a Pharmacy Communications I Counseling Guideline, as illustrated in Figure 1, was distributed to students in conjunction with the lecture material.Demonstration videos were used at the conclusion of the lecture to reinforce the major counseling components highlighted in the Counseling Guideline and covered during the lecture.

Figure 1. Pharmacy communications I counseling guidelines
After the lecture and video review, students were provided the opportunity to practice their counseling skills in pairs.Each pair received two mock prescriptions (cephalexin and montelukast) and associated package inserts.
Students were asked to individually prepare one prescription on which to counsel.Approximately thirty minutes were allotted to review the package insert and practice counseling.Faculty rotated throughout the room to observe counseling sessions and provide feedback to students.Once students felt comfortable with their counseling abilities, the class regrouped.Volunteers then demonstrated their counseling sessions in front of the class.Upon completion of each practice counseling session, feedback was solicited from the class to help develop students.Any students interested in practicing via this method could do so.

Video-Recorded Counseling
The next phase of the counseling series involved a counseling practical day scheduled for the following week.After the first week's lecture/counseling practice day, a mock prescription (metoprolol succinate XL) and associated PI were posted on the university course management system, Sakai, for students to review in preparation for their counseling practical.The second class period (counseling practical day) consisted of a video-recorded counseling session and evaluators' assessments of students' counseling skills and clinical knowledge.Evaluators acted as patients and assessed students by utilizing a standardized grading form.The grading form incorporated the items listed in the Pharmacy Communications I Counseling Guideline.Grading forms also contained standardized responses for instructors based on the questions asked by students to maintain consistency among evaluators.Counseling practical sessions were limited to a maximum of ten minutes with a goal for students to counsel within six minutes.Although additional counseling session practicals were conducted throughout the remainder of the semester, none were recorded.
Upon completion of the practical session, counseling videos were made available for review.Students did not receive feedback or a grade at that time.Only individual students and faculty had access to their video.Students were instructed to review and critique their videos over the following week and prepare a list of strengths, areas needing improvement, and an action plan for self-improvement.This feedback was to be based off of their communication skills and the items listed in the Counseling Guideline.

Study Survey and Feedback
The final class period for the introduction to counseling portion of the course was the feedback/grade review day.Students completed an anonymous, eight-question, paper-based survey (Figure 2) prior to discussing their video-recorded counseling session.The survey evaluated the impact of the video review on their perceived communication skills and the activity's value.Six questions addressed various aspects of communication, such as eye contact, facial expressions, body position, and quality of speech; one question addressed students' opinion of the self-assessment activity in developing their communication skills.Additional comments regarding the self-assessment process were solicited through a final open-ended question.Convenience sampling was used to collect data.

Figure 2. Video self-evaluation survey
Evaluation of the activity's utility was measured by a change in student perception prior to and after counseling video review.Those who noted that video review led them to believe they did better than originally thought, were rated as initially underestimating their performance.An example of this would be a student who believed that he/she had poor eye contact prior to reviewing the recorded counseling session and then rated his/her eye contact higher after reviewing the recorded counseling session video.Those who indicated that they believed they did worse than originally thought were noted as initially overestimating their performance.Any change from the initial perceived performance, regardless of overestimation or underestimation, was considered a reflection of the self-assessment providing a more accurate perception of the student's performance.
Once the survey was completed, students were matched up with their evaluators to discuss the self-assessment and receive feedback and a grade.Evaluators began the session by reviewing the strengths, areas needing improvement, and action plan for improvement.Students were also asked to describe how reviewing their video impacted their performance perception.Any remaining feedback in areas not covered during the student's self-evaluation was then addressed by the evaluator.Students received full credit for the assignment regardless of their performance but were told their final raw score.Feedback regarding students' performances, comments, and self-evaluations was obtained from evaluators at the conclusion of the review session.In addition, evaluators provided insight into the activity's design, flow, and implementation for future semesters.

Ethics and Procedures
A notice of exemption from IRB review was granted by the Office of Research and Sponsored Programs at Rutgers, The State University of New Jersey.

Participants' Characteristics
The Pharmacy Communications I course included 198 P3 students.Approximately 95% (n=188) of students completed the self-assessment survey.Although demographic data were not collected with the in-class survey, Table 1

Survey Data
Figure 1 illustrates the impact of self-assessment on the perceptions of student communication skills.Overall, 95.6% (173/181) of students agreed (15.5%) or strongly agreed (80.1%) that the self-assessment process was valuable in the development of their communication skills.Only one student (0.6%) disagreed and 3.9% of students were neutral regarding the activity's value.Eye contact was identified as the skill with the greatest difference in assessment perception after reviewing the recorded counseling session in approximately 50% of students; 35.1% underestimated and 14.4% overestimated this ability.Approximately 40% of students noted changes in perceptions regarding appropriate rate of speech (42.1%), voice volume (39.8%), and facial expressions (37.2%).Body positioning/gestures and voice pitch were associated with the fewest changes in perception, at approximately 33%.No comments were provided for the final open-ended question regarding additional feedback pertaining to the self-assessment process.

Discussion
In this study, a counseling activity with video review and self-assessment components capable of assisting students in identifying changes in communication skills perceptions was successfully implemented in a Pharmacy Communications class.Consistent with previous studies (Del Mar & Isaacs, 1992;Nilsen & Baerheim, 2005;Yoo et al., 2009), a majority of students agreed that the self-assessment exercise was beneficial in the development of their communication skills.
Similar to findings by Mort and Hansen (2010), a large percentage of students in our study changed their score after reviewing their video-recorded counseling sessions.In Mort and Hansen's (2010) assessment, pre-video self-assessments were scored significantly (p<0.0001)lower than the post-video assessments.Our study found that approximately 39% of students had a perceived change in performance after watching their videos with 27% and 13% indicating they underestimated or overestimated their nonverbal communication skills, respectively.These changes are reflective of research by Zick et al. (2007) which indicates that video review is effective at providing valuable insight into students' strengths and weaknesses.Their findings illustrate that approximately 18% of students are more aware of their strengths and weaknesses by observing the impact of their interactions with simulated patients (Zick et al., 2007).
Results from our study indicate a varying degree of change in the verbal and nonverbal communication skills assessed.The greatest difference in post-video review assessment was associated with the extent of eye contact during the counseling session.Since this form of non-verbal communication may have been difficult to assess without self-observation, viewing the video could have better illustrated the student's ability to make eye contact.Underestimations of eye contact may have been related to perceived nervousness or referring to the prescription blank for counseling information.Several studies have described the varying degrees of anxiety that are associated with the anticipation of being video-recorded during simulated patient interactions (Del Mar & Isaacs, 1992;Lindon-Morris & Laidlaw, 2014;Nilsen & Baerheim, 2005).Del Mar and Isaacs (1992) reported that approximately 25% of students whose consultation skills were video-recorded felt intimidated by the camera.
The simple presence of a video camera may have impacted students' abilities to accurately recall and evaluate their counseling performance prior to watching their video.
The category of body position/gesture produced the least amount of change in self-assessment evaluations of nonverbal communication skills.Variability in this category may have been reduced due to room design and space limitations.Since the counseling rooms are small with an immovable table between the student and evaluator, students were directed to sit in a specific chair and location within the room.This predetermined seating arrangement and limited space for repositioning predisposed students to sit in an appropriate manner.In addition, variations in hand gestures may have been reduced since students were provided the opportunity to use notes or the paper prescription handout available during the session.This may have occupied their hands and minimized distractive movement.
It has been often found that students have a high grade orientation, described as a sole focus on the obtainment of grades as a function of learning (Janzow & Eison, 1990).This may become a barrier when discussing and receiving constructive feedback.Using self-assessment and video-recorded reviews of activities may help facilitate discussions and shift the attention away from grades and towards improving students' communication and clinical skills.Many faculty evaluators mentioned that beginning the feedback sessions with the student self-assessments appeared to improve constructive dialogue.Evaluators also indicated it allowed them to focus on how communication skills could be improved while avoiding a grade-driven discussion.

Limitations
Some limitations identified throughout the self-assessment activity may impact its use in practice.As reported by Ozcakar et al. (2009), more time is required for feedback associated with videotaping compared to only verbal feedback.In addition, resources including video equipment, a course management system, and use of class time were all important factors considered when implementing this activity.Four hours of class time over two weeks were dedicated to recording and uploading videos, completing evaluations, and providing students feedback.
Students also spent time reviewing their counseling sessions and preparing self-evaluations outside of the classroom.
Measures were not implemented to ensure that students reviewed their videos prior to developing their self-assessments and completing the survey.Students may have completed the self-evaluation and surveys without watching their counseling videos.Based on class size and available resources, study activities such as video reviews, self-evaluations, survey completion, and feedback discussions could not be performed immediately after the counseling session.As a result, the self-assessment survey was completed the following week and assessed students' perceptions of change based on memory rather than a direct comparison of two separate self-assessment evaluations (pre-and post-) performed in proximity to the counseling session.Adding a pre-video assessment could establish a baseline to which post-video assessments could be compared.This could also help evaluators provide feedback immediately following the evaluation rather than taking notes for future discussions.
Unlike previous studies involving video-recorded activities (Mort & Hansen, 2010;Ozcakar et al., 2009), this evaluation did not directly compare students' self-assessments to evaluators' assessments.Subsequently, longitudinal assessments throughout the semester were not performed.Since the activity was conducted only once, at the beginning of the semester, its impact on student development over the semester and comparison at the end of semester could not be determined.Including grade comparisons within the activity and throughout the semester to determine if students identified similar communication skill strengths and weaknesses could be incorporated into future studies.In addition, faculty evaluators were not directly surveyed for their feedback on the self-assessment activity and student performance but anecdotal comments were obtained.
Finally, student demographic data were not collected or assessed with respect to responses provided.Since 95% of the class completed the survey, data from the AACP 2011 Enrollment survey for the Rutgers P3 students was included as an overall overview of student characteristics.Correlations among student demographics, responses, and changes in perceptions could not be made.

Conclusion
Self-assessment of video-recorded counseling sessions impacted students' perceptions of their verbal and non-verbal communication skills.Student scores changed after video review, with the most notable change occurring in the category of eye contact.Overall, the majority of students found the video recording to be a beneficial tool in their development.Incorporating self-assessment activities, such as a counseling video review, may assist in identifying more accurate perceptions of communication skills while focusing more attention on student development and performance.A video-recorded counseling session may also help improve students' awareness of isolated communication skills such as eye contact, speech rate, voice volume, and facial expressions.

Figure 3 .
Figure 3. Impact of video self-assessment on initial perceptions of student communication skills (n=188) highlights the Rutgers P3 class data from the American Association of Colleges of Pharmacy (AACP) 2011 Enrollment Survey.