Changes in Study Strategies of Medical Students between Basic Science Courses and Clerkships Are Associated with Performance

We tested the hypothesis that medical students change their study strategies when transitioning from basic science courses to clerkships, and that their study practices are associated with performance scores. Factor scores for three approaches to studying (construction, rote, and review) generated from student (n=150) responses to a questionnaire were correlated to examination and clinical performance scores. Composite factor scores were compared using a paired t-test and sign test to examine changes in study practices as students transitioned from basic science courses to clerkships. The construction approach to studying was more likely to have a positive and stronger relationship to examination scores in both courses and clerkships, but showed no significant associations with clinical performance scores. Our analyses indicated that students are more likely to increase their use of study practices associated with construction of knowledge as they transition from courses to clerkships. Although learning is a complex endeavor, students employing construction study strategies are more likely to outperform their peers who rely mostly on rote and review practices. Transitioning from basic science courses to the clerkships students tend to utilize more construction study practices suggesting that students are responsive to their learning environments when selecting study strategies.

How a student chooses to study influences the manner in which information is processed and subsequently encoded into longer-term memory.This process influences how the information is represented mentally and retrieved for later use (Anderson, 1995;Ormrod, 2007;Driscoll, 2005).Often, students choose to use study methods that capitalize on rehearsing, repetition or forced associations.These strategies emphasize the learning of discrete pieces of information in original forms that are not connected to one another in a meaningful manner during encoding, and are retrieved as separate pieces of information.Strategies that emphasize these methods tend to be grouped together and referred to as rote learning (Ormrod, 2007).Study strategies that capitalize on creation, elaboration, structured organization, visual representations, personal connections, and emphasize actively making new meaning of the information tend to be grouped together and viewed as constructivist learning (McNulty, et al., 2012;Ormrod, 2007;Driscoll, 2005).Thus, the methods of processing information a medical student selects when preparing for a basic science course or clerkship reflects the nature of learning that occurs in each environment.Feighny, 1995;Crooks, Winter 1988;Newble & Entwistle, 1986;Richardson, Dawson, Sadlo, Jenkins, & Mcinnes, 2007), with some researchers suggesting that a student's existing and implicit perspectives about which practices and strategies are most effective for learning independent of the context mitigate their degree of responsiveness to the learning context (Arnold & Feighny, 1995;Edmunds & Richardson, 2009).While some studies have examined students' study habits over a span of time (Al Kadri, et al, 2011;Al-Kadri et al., 2012;Arnold & Feighny, 1995) we located only one study examining study habits where the time span included both basic science course work and clinical rotations (Arnold & Feighny, 1995).Arnold and Feighny (1995) used the Lancaster Approaches to Studying Inventory (LASI) to assess students' study habits across four years.Their scoring of individual items produced seven specific learning dimensions.Four of these dimensions (reproducing orientation, meaning orientation, comprehension, and globetrotting) had specific connections to the information processing activities students used when studying.Their factor analysis of student data indicated that the student's scores could be reduced to three factors of learning.One factor reflected student motivation and two factors reflected information processing activities, which they distinguished as either a surface approach or a deep approach.The surface approach included items scored for the reproduction orientation "rote memory" and the globetrotting a "fragmented approach to studying".The deep approach included items scored for comprehension dimension described as "a search for connections and relationships in the materials" and meaning orientation dimension "a search for deep understanding of the subject" (p.716).
Using regression analysis the authors examined the predicative value between the seven dimensions and student performance, operationalized as GPA in years 1-4, individual grades in specific science courses, and clinical rotation rating scores for two cohorts of students.Results indicated that higher scores on achievement motivation and lower scores on reproduction orientation and globetrotting (i.e., surface approach) had meaningful predictive value for higher performance in academic courses, however only one cohort score on these dimensions had predictive value for clinical ratings.
In previous research, a factor analysis of a self-reported study practices questionnaire indicated student practices fell into three main study approaches for their basic science courses and reflected ways of processing information when studying for these courses (McNulty, et al., 2012).Two of these factors, rote and construction approaches, represented study practices associated with the forms of learning described by Driscoll (2005) and Ormrod (2007).The third factor was labeled review as the practice used processes of re-examination of course information.
The rote approach included the use of memorization, flashcards, mnemonics and practice tests.These practices represent rehearsal, forced associations, and the discrete and separate encoding of information described by rote learning practices.This approach is similar to the learning dimensions reproduction orientation and globetrotting that comprise the surface learning construct reported by Arnold and Feighny (1995).The construction approach included the use of concept maps, life examples, changing original information into own words, and distributed learning by building on previous knowledge.These strategies emphasized the structural organization, visual representation, and personalization of information described in the constructivist learning practices and is similar to the comprehension and meaning orientation learning dimensions that comprise the deep learning construct reported by Arnold and Feighny (1995).The review approach included reviewing notes and class handouts as well as highlighting text.This approach represents a process of reexamining existing materials and may be used in conjunction with the other two approaches.McNulty et. al (2012) examined the relationship between the three factors and students exam performance in basic science courses.The results from that study indicated patterns of positive and stronger relationships between exam performance and the construction approach across basic science courses, while relationship between the rote approach and exam performance showed a pattern of weak and negative relationships across the courses (McNulty, et al., 2012).
Although research exists that examines student approaches for studying while involved in courses (McNulty, et al., 2012;Onwuegbuzie et al., 2001;Pandey & Zimitat, 2007), little information exists on how students study while in their clerkships (Al Kadri, et al., 2011;Al-Kadri et al., 2012;Arnold & Feighny, 1995), the relationship between study strategies and clerkship examination performance, or if students adjust their study strategies as they transition from the basic science curriculum into the clinical setting.This gap in knowledge regarding medical students study practices in clerkships and their transition from courses to clerkships led to the following research questions: Which study habits change as students move from basic science courses into clerkships?
What is the relationship between study factor scores and students' course and clerkship exam scores?Is there a difference between the study factor scores of students in their basic science courses as compared to their clerkship courses?

Methods
Data for this study were collected from a cohort of medical students (n=150).Students in the cohort were asked to complete a survey on study strategies after each of the basic science courses throughout their second year and after each of the clinical clerkships in their third year.The data were derived from the five basic science courses; Mechanisms of Human Disease (MDH1 and MDH2), Pharmacology and Therapeutics (Pharm1, and Pharm2) and Neuroscience (Neuro); as well as data from seven clerkships Family Medicine (FM), Psychiatry (Psych), Obstetrics and Gynecology (Ob-Gyn), Pediatrics (Peds), Medicine (Med), Surgery (Surgy), and Neurology (Neurol).For those courses that spanned the full year (e.g., MHD), each semester was treated as a separate course for statistical analysis because students received a separate letter grade for each semester.The survey data were collected as part of the confidential end-of-course or clerkship evaluations students were asked to complete.Results of examinations were obtained from the school's grading system.All data were entered into Excel spreadsheets and student identifiers deleted prior to analyses.

Survey Instrument
The survey on study strategies was designed using constructs developed by Karpicke et al. (2009).The survey instructed students to rate, on a five point Likert scale, the degree to which they used individual study strategies throughout each course or clerkship (McNulty, et al., 2012).Previous research using this instrument (McNulty, et al., 2012) indicated that three factor scores could be calculated using groups of individual study strategy scores.The three factor scores identified by McNulty et al. (2012) aligned with three different methods of studying and encoding information and represented learning approaches (Driscoll, 2005;Ormrod, 2007).These approaches were labeled rote, construction, and review as they represented the information processing activities associated with the individual practices.For more detailed information on the factor analysis and the instrument, see McNulty et al. (2012).
For the purpose of this study, individual factor scores for each approach were calculated from student responses to the survey on study strategies for each course and clerkship.In addition, course composite factor scores for each of the three approaches (i.e., rote, construction, and review) were calculated by averaging the mean factor scores for all basic science courses.Clerkship composite scores for the three factors were calculated in the same manner.

Analysis
To examine the change between students' study strategies from basic science courses to clerkships, the mean scores across all courses and clerkships were calculated for each study habit.To analyze the relationships between factor scores and performance, students' examination scores for each course and clerkship were correlated to the students' three factor scores using Pearson's r; a total of thirty-three correlations were calculated.Performances were based on the multiple-choice examinations given in all courses and clerkships and clinical performance scores given in each of the clerkships.
To examine the significance of change of factor scores from basic science courses to clerkships, the composite factor scores from all courses and clerkships were compared using a paired t-test to determine significance on a group level.The paired t-test was followed by a paired non-parametric sign test to determine the number of students whose study composite factor scores changed between course and clerkships.A total of three paired t-tests and sign tests were calculated.

Institutional Review Board
The study design was reviewed by the Institutional Review Board at the Stritch School of Medicine and was exempted from further review and monitoring.

Results
The first research question addressed changes in study strategies as students moved from their basic science courses into clerkships.To answer this, we calculated the mean score for each study habit across all basic science courses and clerkships and compared the differences between these means.Table 1 illustrates the ranges and means for individual study strategies calculated for each of the basic science courses and clerkships.Study strategies where the mean values declined by >0.5 (1-5 Likert scale) included reviewing handouts and notes, use of flashcards, mnemonics, lecture videos and concept maps, as well as group study and memorizing.The only study strategy that had a mean increase of >0.5 (1-5 Likert scale) was the use of real-life examples.These results help to validate the survey instrument.For example, the average decline in student reliance on lectures videos for studying in the clerkships relates to the reduction in lectures during this part of the curriculum.Similarly, there were fewer flashcard applications developed for the clerkships.The ranges of means for study strategies across courses and clerkships tended to be relatively narrow.One exception was the use of practice tests by students in the clerkships, which ranged from 1.8 (Neurology) to 4.3 (Ob/Gyn).Students in the Ob/Gyn clerkship utilized practice tests generated by their clerkship organization (APGO).Collectively, the data from Table 1 suggests that students tend to take advantage of resources to promote learning and studying when made available in both courses and clerkships.

Description:
The N for each course or clerkship ranges from 115-138.This range reflects the number of students that completed the study strategy survey upon completion of each course and clerkship.The means below represent the average of all student responses for all courses and all clerkships for each study strategy.Study strategies are grouped according to study approach: Review, Rote, Constructive, and Other.
(+) indicates a 0.5 or greater increase from the basic science courses to clerkships (-) indicates a 0.5 or greater decrease from the basic science courses to clerkships To answer the second research question, "What is the relationship between study factor scores and students' course and clerkship examination scores?", study factor scores were calculated for each student for the three study factors across all courses and clerkships.Factor scores were then correlated to students' course and clerkship examination scores.Table 2 shows the results of these correlations, which indicate weak and negative patterns of correlations between examination scores and Rote factor scores.The same pattern is apparent in the correlations between Review factor scores and examination scores.However, the correlations between Constructive factor scores and examination scores showed stronger, positive relationships for both courses and clerkships, with statistically significant correlations in the MHD1, MHD2, and Neuroscience courses, and statistically significant correlations for Medicine, Ob/Gyn, and Pediatric clerkships.Comparisons between the basic sciences and clerkships were possible only through analysis of the multiple-choice examinations used throughout the curriculum.In the clerkships, a second important component of overall grades is the faculty evaluation of student clinical performances, which involves considerable subjectivity.The results of correlations of clinical performance scores to factor scores did not reveal any of the trends seen with the examination scores (Table 2).In fact, the only significant, positive correlation was with the Rote factor score in the Ob/Gyn clerkship.These differences in observations between clerkship examination scores and clinical performances are related to the absence of any significant correlations between individual student scores on their examinations and clinical performances (data not shown).
To answer the third research question "Is there a difference between the study factor scores of students in their basic science courses as compared to their clerkships?",three paired t-tests were conducted comparing the three composite factor course scores with their paired composite factor clerkship score.A review of the data resulted in N = 123 cases with sufficient data from their individual course and clerkship factor scores to produce adequate paired composite factor scores for use in compared t-tests.Table 3 presents results of the paired t-test showing significant differences for all three of the factor scores.The average composite factor scores for rote and review decreased from course to clerkship and the average composite factor score for construction increased from course to clerkship.Although, the paired t-test results indicated changes in the mean factor scores as students moved from course work to clerkship, the t-tests did not provide any information on the percentage of students who changed their study approach or the direction of their change from courses to clerkships.

N =123
Employing the same paired data set, a sign test was conducted to calculate if the number of students who altered their study approaches was significant, to determine the percentage of students who changed, and in which direction.Table 4 displays the results of the sign test and displays the number of students whose study factor score changed along with the average increase or decrease of change.Results of the sign test indicated that most students increased their Construction factor score, while most students decreased their Review and Rote factor score.The results for all three factor score sign tests were significant at p = .001.Our results further indicated that more students' Rote and Review approach scores decreased as they transitioned from courses to clerkships and more students' Construction approach scores increased during this transition.In order to examine the degree to which the Composite factor scores increased or decreased, we calculated the average change in each of the three composite factor scores for those students whose scores either increased or decreased.Table 4 shows the mean increase and decrease for those students whose mean Composite factor score changed from courses to clerkship.It is important note that the Composite factor scores range from 1-5 and an average change of 0.80 indicates almost a full point change.On average, the amount of score change for those students whose Rote and Review factor scores reduced from course to clerkship was greater than the amount of change for those whose Review and Rote factor scores increased.The opposite pattern emerged for the Constructive factor score.In other words, on average, the amount of score change for those students whose Construction factor scores increased from course to clerkship was greater than the amount of change for those whose Construction factor scores decreased (Table 4).

Discussion
The methods and practices a student uses to study influences the way in which information is coded into long-term memory and how easily that information can be retrieved and used at a later date (Anderson, 1995;Ormrod, 2007).This study provided new and important results regarding the nature of the study strategies medical students utilize and their individual performances in both basic science courses and clerkships.Our findings indicate that employing study strategies that foster the organization, continued elaboration, and creation of meaningful understandings with information are more likely to result in better course and clerkship performances, while the use of more rehearsal or discreet learning, or the reexamination of materials do not.
Our conclusions were similar to those reported by McNulty et al. (2012), who found a positive relationship between constructive approaches to studying and performance in first and second year basic science courses for medical students.Conversely, there were weaker and more negative relationships for either rote or review approaches to studying on performances these results seem to substantiate previous findings that rote learning is negatively related to academic performance (Arnold & Feighny, 1995).Extending these results, we found similar patterns of stronger and mostly positive correlations between constructive study strategies and performances in clerkship examinations.However no relationships were found between study approaches and clinical ratings.As did Arnold and Feighny (1995), we conclude that the subjectivity of the clinical ratings limits their use for examining relationships between learning approaches and performance.In general, approaches to studying that require medical students to structure, make connections and generate meaning are more likely to foster stronger learning and performance in both courses and clerkships.Based on these findings, medical schools should consider fostering study strategies that align with more constructive approaches to learning for both basic science courses and clerkships.
For medical students, transitioning from basic science courses to clerkships presents them with new learning environments, which change from a lecture/small group environment to one of authentic practice and apprenticeship.Several researchers reported that learners tend to adapt their learning to match the nature of the assessment they expect or experience (Al Kadri, et al., 2011;Al-Kadri et al., 2012;Arnold & Feighny, 1995;Crooks, 1988;Newble & Entwistle, 1986;Richardson et al., 2007).If a student expects assessments to focus on the recall of factual information over the application, synthesis or analysis of information, they rely on more rote-based approaches to learning.If the assessments require the application of information or the evaluation of information, they tend to employ constructive based approaches to learning.Research suggests that learners respond and adapt to their learning context (Arnold & Feighny, 1995;Crooks, 1988;Newble & Entwistle, 1986;Richardson et al., 2007).Results from the present study support this notion of student adaptation and responsiveness, but may not be as influenced by the type of exam assessment.While the exam assessment system (i.e., multiple choice tests) remained the same in clerkships, the authentic nature of the context appears to have influenced the preferred methods of studying for students in this report.As medical students moved from classroom-learning into the authentic settings of their clerkships, they reduced their rote and review approaches to learning, and increased their construction approach in order to match the authentic nature of the clerkships.While these results support previous models of student adaptation in learning (Arnold & Feighny, 1995;Newble & Entwistle, 1986) it is difficult to determine if students might forgo their "baseline approaches" (Arnold & Feighny, 1995) to learning, and adopt new learning approaches as they progress through a program of study or do students simply respond to context?
The study strategies associated with the construction approach remained the same for distributed review and explaining in own words as students transitioned into new learning settings, suggesting that they perceived the relative value of ongoing learning that builds on previous knowledge, as well as the value of making learning more meaningful through their own words.It is noteworthy that use of life examples increased from basic science courses to clerkships.This shift is easily explained as clerkships present medical students with real life examples as part of their learning processes and further indicated that students are responsive and adaptive to the learning context.
Surprisingly, concept mapping was the least used construction study strategy in courses and diminished as students moved into the clerkships.The value of concept maps in medical education is well researched (Gonzalez, Palencia, Umana, Galindo, & Villafrade, 2008;Rendas, Fonseca, & Pinto, 2006;Torre et al., 2007;Weiss & Levison, 2000;West & Sadoski, 2011).Daley and Torre's (Daley & Torre, 2010) review highlights the value of concept maps in both basic science courses and clinical settings, as well as the ability of the strategy to bridge the learning between the two settings.The authors (Daley & Torre, 2010) indicated that this strategy benefits students in problem based learning, collaborative learning, critical reasoning and establishing relationships between concepts.The act of building a concept map mimics the underlying cognitive processes a student engages in when constructing information in a meaningful manner, and facilitates the use of the information at a later time (Ausubel, 2000).The low implementation of this strategy among students in our study suggests a lack of awareness of its value when studying for both science courses and clerkships.This may be related in part to research suggesting that using concept maps does not provide additional performance value on assessments such as multiple-choice tests (Gonzalez et al., 2008), which was the primary method of assessment showing relationships in our study.Because concept maps have importance to problem solving assessments, and authentic practice (Gonzalez et al., 2008), the purposeful incorporation and promotion of concept maps as a learning strategy in classrooms and clinical settings is warranted for training towards the professional nature of medicine, which requires cognitive dispositions of critical thinking and problem solving among its members.
The decline in use of videos from basic science courses to clerkships is related to the decrease in availability of lecture videos during the clerkships.During the basic science part of the curriculum, the largest portion of content is delivered in lectures, which are recorded and made available to students for review.Once students move into the clerkship portion of their studies, the curriculum is less lecture-driven though some clerkships may include instructional videos as supplementary resources.This change in the use of videos for study during the clerkships further emphasizes the responsive nature of students to adapt to the availability of resources to assist with their learning and agrees with other research indicating students use available resources they perceive as directly connected to the formal curriculum (Zhang, Peterson, & Ozolins, 2011).
An important limitation of our study is the reliance on self-reporting of study strategies by the students.However, several observations strengthen the reliability of our survey results.For example, the average decline in student reliance on lectures videos for studying in the clerkships related to the reduction in lectures during this part of the curriculum.Similarly, declines in students' acknowledgments of their use of flashcards and study groups coincided with fewer flashcard applications developed for the clerkships and the reduction in small group exercises.Bow, et.al. (2013), found that a crowdsourcing model which, allowed for the collaborative creation and editing of flashcards was positively related to students performance on exams.It is important to note is that the students themselves had the ability to create, expand on, and correct information used to develop the questions for the flash cards.This indicates that students were more active in the creation and elaboration of the knowledge used in the system, and suggests the crowdsourcing system had elements of constructivist learning (Driscoll, 2005;Ormrod, 2007), even though flashcards are typically associated with rote learning.
In conclusion, learning is a complex process, requiring students to engage in practices of recognizing, encoding, storing and retrieving information (Anderson, 1995;Ormrod, 2007).The strategies students select to facilitate the learning process can have significant impact on their performance.In addition, students make decisions about the learning process in relation to the characteristics of the learning environment, including the purpose of the learning, how the information will be used, and even the nature of the assessment.The results of our study indicate that medical students do make choices about the ways they approach studying; exhibiting a responsive nature by changing their approaches as they change learning environments and are required to apply their learning in different ways.Additionally, the use of constructive practices has a stronger and more positive relationship with students' performances across these learning environments.While this study does not provide a causal effect understanding of this relationship, it supports previous research indicating the relationship between methods of learning and performance of medical students (McNulty, et al., 2012;Weinstein & Gipple, 1974;West & Sadoski, 2011), and the connections between specific study strategies and performance of medical students (Al Kadri, et al., 2011;McNulty, et al., 2012;Daley & Torre, 2010;Gonzalez et al., 2008;Wenger, Hobbs, Williams, Hays, & Ducatman, 2009).Further research could explore how prospectively incorporating these construction approaches into the pedagogical practices within courses and clerkships impacts student learning and performance.The institutional Teaching and Learning Center facilitates the implementation of strategies to influence changes in student study habits and pedagogies for self-learning.Specific examples include: employing real life examples as a part of basic science courses, having students construct concept maps to organize and build connections across courses and into clerkships, having students create presentations on course and clerkship information in their own words, and having students communicate the explicit connections between their existing knowledge and new knowledge presented in courses and clerkships.

Table 1 .
Means of individual study strategies for basic science courses and clerkships

Table 2 .
Relationships of study methods versus exam performance illustrated by r values from Pearson r analysis by course *Significant at p = .05**Significant at p = .01

Table 3 .
Paired t-test for study composite factor scores

Table 4 .
Sign test differences between course and clerkship composite factor scores