Study Strategies Are Associated with Performance in Basic Science Courses in the Medical Curriculum

We investigated the study strategies of first and second year medical students and tested the associations between study habits and performances in their basic science courses. Upon completion of every basic science course, students completed a survey ranking the study strategies they utilized throughout each course. Results of a principle component analysis showed that study strategies clustered into one of three study factors: “rote” learning, “constructive” learning, or “review” learning. Each of these study factors comprised related study strategies. Students tended to use “constructive” strategies predominantly, but altered their study habits based on content delivered in specific courses. Trends emerged indicating negative correlations for “rote” learning and course performance whereas there were positive correlations for “constructive” learning and course performance. Courses where “constructive” learning had the greatest effects also tended to have the greatest number of questions that required “constructive” reasoning on the final exam.


Introduction
The study skills and strategies of students (including medical students) have been the subject of numerous investigations (Amin, Tani, Eng, Samarasekara, & Huak, 2009;Barker & Olson, 1997;Karpicke, Butler, & Roediger, 2009;McConnell, Regehr, Wood, & Eva, 2011;Newble & Gordon, 1985;Pandey & Zimitat, 2007;Reid, Duvall, & Evans, 2005;Salamonson, Everett, Koch, Wilson, & Davidson, 2009;P. Weinstein & Gipple, 1974;Wenger, Hobbs, Williams, Hays, & Ducatman, 2009;West & Sadoski, 2011), but there is little evidence of associations between study strategies and performance in medical courses (P.Weinstein & Gipple, 1974;West & Sadoski, 2011).Definitions of learning are often characterized by two components, a change in behavior or mental representation, and an experience that facilitates that change (Driscoll, 2005;Ormrod, 2004).Cognitive theories of learning include activities or methods used by an individual to encode information into long-term memory in the category of experiences that produce changes in mental representations and thereby constitute learning (Anderson, 1995).Accordingly, the activities an individual engages in to promote encoding while studying course content represent learning strategies of the individual.Ormrod (2004) indicated that the processes or strategies employed by a learner while studying influences how the information is stored and later retrieved, and she proposed that some study processes promoted better learning.structures."Constructive" processes imply that the learner transforms the information from its original form and constructs a new representation of the information in long-term memory, which includes linking the information into existing knowledge structures.Constructive processes include strategies for encoding information through the elaboration of prior knowledge, structuring or organizing information, personalizing information to make it meaningful, and creating visual representations of information (Driscoll, 2005;Ormrod, 2004).Ormrod advocated that students utilize constructive study strategies."To learn classroom subject matter effectively, students should instead develop study strategies that involve meaningful learning, internal organization, and elaboration…they can be encouraged to state ideas in their own words, generate their own examples of an idea, make connections between new concepts and their past experiences, and draw logical inferences from data they receive" (pg.227).Given the cognitive science views of learning as an encoding process and the value placed on constructive strategies, the expectation is that students who utilize constructive strategies when studying learn more effectively and perform better on assessments.While evidence supports the value of using constructive practices, students often resist using constructive strategies, assuming that rote learning strategies are more effective (Ormrod, 2004).
The perceived value of rote learning by students is fostered when assessments and assignments emphasize recall or recognition of factual information, and do not require students to integrate or use information in meaningful ways (Crooks, 1988).Based on this, one can assume that students whose previous learning experiences involved courses where assessments emphasized recall and recognition of factual information, would be more inclined to use rote learning strategies while studying and may not employ constructive strategies when course content and assessments required them to integrate or use information in a meaningful way.The present study evaluated the hypothesis that study strategies of medical students are associated with their performances in basic science courses.

Methods
The data were collected from first year (MS1; n=151) and second year (MS2; n=150) medical students at the Stritch School of Medicine (Loyola University Chicago).All students enrolled in each of the nine ( 9) separate basic science courses that were included in our study (i.e., 150 or 151 students per course).Four (4) of the courses spanned a full year and students received a letter or pass/fail grade after each semester.For the statistical analyses, each of the semesters was treated as a separate course.

Survey on Study Strategies
A survey of study strategies (see Appendix 1) was based on one developed by Karpicke et al. (2009).The number of students completing the survey upon completion of each basic science course taken during years 1 and 2 of the medical curriculum ranged from 129-151 students.The survey asked students to rate on a Likert scale (1-5) the degree to which they used specific study strategies throughout each course (see Appendix 1).The surveys were distributed online as part of the end-of-course evaluations students were required to complete for each course.All surveys were accessed through the school portal, which required a login id and password.

Statistical Analyses
Data from the surveys were entered into a database and all student identifiers were deleted.An exploratory factor analysis was conducted to examine the internal structure of the study habits questionnaire and determine the existence of underlying constructs representing how students approach studying for courses.Prior to conducting the factor analysis a correlation matrix was generated to examine the level of multi-collinearity between the individual study methods.The correlations between the 13 study methods ranged from r = .085to .470indicating a low level of multi-collinearity among the variables.Barlett's measure was significant p = .000indicating that some relationships existed among the student study methods.Accordingly, factor analysis was appropriate for examining the underlying constructs or approaches to studying.The test for adequate sample size indicated a good sample size for determining underling structure with the value of the KMO = .815.The anti-image correlations ranged from r = .786to .893 with all values above the recommend 0.5 indicating that all variables could be included in the factor analysis.Factor analysis using a principle component extraction method was conducted where component factors with Eigen values of 1 or greater were maintained for analysis.A varimax rotation was applied to the solution to increase the dispersion of variables onto factors to maximize the interpretation of the constructs.Once factor scores were calculated for students in each of the nine courses, separate regression analysis were run to examine how these study factors related to the weighted scores for each course.

Institutional Review Board
The study design was reviewed by the Institutional Review Board at SSOM and exempted from further review and monitoring.

Results
The results of our survey (Table 1a, b) revealed high ratings given by students for use of handouts and class notes for studying in all courses except the PCM (Patient Centered Medicine) courses.Students also tended to distribute their review of course material throughout the courses and they relied heavily on memorization, again with the exception of the PCM courses.On the other hand, students tended to give low ratings to use of lecture videos for studying, and they did not rely extensively on study groups except in the anatomy course.There was variability in the survey results for study strategies from course-to-course, which was predictable based on course content and course resources.For instance, the students indicated greater use of flash cards in the anatomy and pharmacology courses, which were the only courses that provided online flashcards as supplemental resources.Similarly, provision of on-line self-tests in the anatomy course would account for the higher ratings for self-testing given by students in that course.
The principle component analysis resulted in a three component factor solution for the questionnaire.The three components together accounted for 50% of the explained variance after the matrix was rotated, with component 1 accounting for 17.5% of the variance, component 2 accounting for 17.1% and component 3 accounting for 15.4%.Stevens (2002) suggested a cutoff point of .40 when selecting variables to load on a component factor.
For this study we selected a more conservative cutoff of .50 in order to maximize the unique contribution of each study method.Study methods with loading values below .50 were not considered to load on a component.Table 2 shows the loading values for the study methods that were .50 or greater for each of the 3 components.Empty cells (e.g., watching lecture videos) indicate the loading value for that study method was below .50.
Component 1 was defined as a "rote" learning since study strategies (i.e., mnemonics, flashcards, practice tests, and memorization) loading on this factor represented study methods that relied more on rehearsal processes for encoding information into memory.Component 2, which we defined as "constructive" learning, included study strategies (i.e., concept maps, life examples, continuous review, and own words) that represented encoding processes that required students to construct meaning through organizing, integrating, and extending the information to generate a deeper understanding of the content.Component 3 was defined as "review" learning approach since study strategies (i.e., handouts, class notes and high lighting) loading on this factor illustrated methods that emphasized encoding process of reexamining or reviewing course resources.
Table 3 shows the percent distributions of students who listed preferentially "rote" vs. "constructive" study strategies for each of the courses.The values represent the ratios of sums of rankings each student gave to the study strategies comprising each group.The majority of students (61-81%) preferentially listed "constructive" study strategies with the exception of two courses, anatomy and pharmacology, where only 28%-36% preferentially used "constructive" study strategies.From 4%-16% of students did not rank a preference for one or the other grouping (i.e., the ratio of their two groupings was 1.0).The "review" factor was eliminated from these analyses based on our determination that this factor represented activities necessary to engage in both "rote" learning and "constructive" learning study strategies.The level of correlations (Pearson's r) from course-to-course for individual students was high.For those students who used predominantly "rote" methods of study the "r" values ranged from 0.23-0.80 with an average of 0.42 (stdev=0.15).Correlations with the smallest effect sizes (r<0.30) were between the PCM course and the basic science courses in cell biology, anatomy and physiology.For students who used predominantly "constructive" methods of study, the level of correlations from course-to-course were similar with those "r" values ranging from 0.24-0.75 with an average of 0.47 (stdev=0.12).In this case, there was only one correlation that had an r<0.30, which was between PCM and anatomy.
After the factors were determined and defined, a factor score for each of the three defined study factors was calculated by summing the response to the study method Likert items associated with each factor (e.g., "rote" approach was calculated by summing Likert ratings for the following study method items: flashcards, memorize, mnemonics, and practice tests).Once factor scores were calculated for students in each of the nine courses, separate regression analyses were run to examine how these study approaches related to the weighted scores for each course.Table 4 shows the standardized beta coefficients for each study approach by each class.The results indicate that for each class the "rote" learning approach was negatively correlated to the weighted score with the beta coefficients ranging from r = -.151 to -.091.These results suggest that the "rote" learning approach to encoding information has a minimal and negative relationship to course performance.The beta coefficients for the "review" learning approach were mixed with the standardized beta coefficients for this study approach ranging from r = -.145 to .143.These results indicate that the "review" approach in general has limited relationship to course performance.While the factor analysis identified specific study methods loading on this component factor, we reasoned that that students would need to review class notes, handouts and highlighted materials, in order to engage in many of the "constructive" learning study strategies, and "rote" learning study strategies that loaded on the other two factors.Given the nature of the specific study methods loading on the "review" learning approach we speculate that this approach may represent an underlying process through which the other two study approaches are accomplished.The courses are listed in the order they were taken.Courses taken in year 1 are separated from courses taken in year 2 by a double line.Cell=Molecular Biology of the Cell and Genetics; Anatomy= Human Anatomy; Physio=Physiology; Immuno=Immunology and Microbiology; MHD=Mechanisms of Human Disease; Neuro=Neuroscience; Pharm=Pharmacology and Therapeutics.
The "constructive" approach was positively correlated to the weighted score in all courses.The beta coefficients for this approach ranged from r = .023to .316,(see table 5) and were significantly related to the weighted score in three courses (pathology, neuroscience, and pharmacology).The results indicate that the "constructive" approach to studying in general is more likely to be related to higher course performance than the other approaches and may have a more significant relationship for specific courses due to the nature of the course content, but raised the suggestion that there may be an influence from the nature of assessment.The answers to these questions could be learned using "rote" study methods.Cell=Molecular Biology of the Cell and Genetics; Anatomy= Human Anatomy; Physio=Physiology; Immuno=Immunology and Microbiology; MHD=Mechanisms of Human Disease; Neuro=Neuroscience; Pharm=Pharmacology and Therapeutics.
The role of course assessments as an explanation for these course specific beta coefficients were suggested by our observation that significant associations between "constructive" study strategies and course performance were found only in year 2 courses (Table 4).Accordingly, examinations for all courses were evaluated by three independent observers who scored each question according to whether it could be answered by knowing a fact (rote) or whether the question required integrative/constructive reasoning.Table 5 shows that examinations in year 1 contained a uniformly high number of factual questions (80-87%).The courses in Year 2, with the exception of Pharmacology 1, contained far fewer "rote" questions (22-70%).

Discussion
The results of our study offer several general conclusions.First, the study strategies of individual medical students in the basic science curriculum tended to cluster into contrasting categories that emphasized either rehearsal/practice strategies or conceptual/organized approaches.A third identifiable "review" group tended to rely more heavily on the use of resources (i.e., class notes, handouts and highlighting).This third group is less distinguishable since "review" learning strategies represent processes inherent to both "rote" and "constructive" study strategies, which accounts for the absence of any trends associated with performance in the courses.Second, students relied more extensively on "constructive" study strategies for most basic science courses.
Finally, there was a significant positive association between the use of "constructive" study strategies by individual students and their performances in the courses.
The groupings of "rote" and "constructive" strategies in the present study correspond closely to earlier groupings of "surface" and "deep" approaches to learning and studying (cf (Reid, et al., 2005)), which used aggregates with different subscales (e.g., time management, purpose, etc.).Reid et al. (2005) reported that students at their medical school tended to have high scores for "deep" approaches, which is consistent with our findings, but contrasts with the report by Newble & Gordon (1985) who found that medical students had higher scores on "surface" approaches, which may be related to differences in student populations and instruments used to measure study strategies.Efforts to increase medical students' "deep" learning strategies through curriculum alignments in a single course (small group tutorials, computer assisted learning programs) resulted in an actual decline (Newble & Gordon, 1985).Our study did not specifically test the effects of interventions on study strategies, but the changes we observed in "rote" vs. "constructive" strategies from course-to-course in a single cohort of students showed that medical students do adjust their study and learning strategies to the curriculum, employing more constructive methods of studying when a course requires a more integrative form of learning.
There is little debate on the notion that academic achievement is associated with study strategies as well as aptitude (cf (Onwuegbuzie, Slate, & Schwartz, 2001)), but there is relatively little direct evidence of this association in medical students.More than 35 years ago, Weinstein and Gipple (1974) reported that achievement of medical students correlated with study skills using an inventory that measured a student's abilities to synthesize and organize materials as well as to discriminate level of importance of content.West and Sadoski (2011) confirmed that study strategies (especially self-testing) are strong predictors of medical student grades in their first semester of medical school.The present study extends these observations to include evidence that combinations of specific study strategies of medical students are associated with performances in different basic science courses, and that constructive approaches are positively related to performance in all course.
The degree to which students used "rote" study strategies was negatively correlated with course performance in all of the basic science courses (see Table 4).This general pattern of weak negative relationships between "rote" learning and course performance, matches with a widely held view in the cognitive sciences that rote learning is an ineffective and inefficient method for learning declarative knowledge (Anderson, 1995).The negative association between "surface" learning strategies and medical student performance on examinations was also reported by Pandey and Zimitat (2007) in their investigation of study strategies in anatomy.The strongest positive association between "constructive" methods of study and performance occurred in our year-long pathology course and also the neuroscience course, where conceptualization and integration are required to address the clinical problems encountered in these courses.Examinations in both of these courses also tended to have the greatest number of questions that required "constructive" reasoning.Given the pattern that constructive processes are better predictors of course performance, medical students should be encouraged to utilize these approaches regardless of course content.Additionally, medical school faculty could consider pedagogical practices that incorporate these strategies into the course instruction, particularly in basic science courses that tend to focus on factual information in order to promote the use of constructive study habits in general.
One particular study strategy that has been investigated extensively is self-testing, or practice retrieval, which has been shown as an effective method for learning (Belluck, 2011;Karpicke, et al., 2009;Larsen, Butler, & Roediger, 2008;Roediger & Butler, 2011;Rohrer & Pashler, 2010;Wenger, et al., 2009).The extent of this study strategy can range from self-motivated recall of information (memory retrieval/rehearsal) to taking multiple-choice practice tests.The wording in our survey did not discriminate between these different uses of this study strategy, but students in our study may have interpreted it to mean the latter.Excluding the Patient Centered Medicine courses, students ranked the use of practice tests as last or next to last as a study strategy in half of the basic science courses.One exception was anatomy which included a number of practice tests as supplemental on-line resources to the course.It is noteworthy that the performance of medical students in a pathology course was significantly correlated with time spent using a practice question book, but not with online practice quizzes (Wenger, et al., 2009).
The reliance on self-reporting of study strategies is a limitation of our study.But differences in course content and methods of delivery helped to validate the survey results.For instance, students rated the strategy of "group study" highest for the human anatomy course, which relies extensively on team activities in the dissection laboratories.Anatomy also had the highest average scores for "memorization" and use of "mnemonics" among the first year courses.Both of these study strategies can be related to course content and the preponderance of new terms and structures.Medical students' perceptions of the importance of memorization in the study of anatomy were also reported by Pandey and Zimitat (2007) in their survey of medical student learning strategies.
Use of flashcards was another study strategy that received overall low ratings with the exception of anatomy and pharmacology, which were the only two courses that utilized an online flashcard application.This provides evidence that while the data was self-reported, students accurately reported their study habits in courses, providing a degree of validity to the study's results and discussion.
Another important limitation is that the survey instrument used in our study did not collect information included in other more detailed questionnaires (e.g., study locations, motivation, time management, etc.).These include the Study Process Questionnaire (Biggs, Kember, & Leung, 2001); the Approaches and Study Skills Inventory for Students (Tait, Entwistle, & McCune, 1998) and the Learning and Study Strategies Inventory (C.E. Weinstein, Schulte, & Palmer, 2010).There were two reasons for using a more abbreviated survey for our study.First, because students are surveyed after each course and clerkship in the 4-year curriculum, we decided that a longer survey would be too burdensome for the students since they are frequently asked to respond to numerous other surveys.Secondly, we sought to include information on resources not included in these other surveys (e.g., use of videos, flash cards).
It was noteworthy that lecture videos had among the lowest overall ratings as useful study strategies.This self-reporting by the medical students substantiates our earlier studies showing lecture videos were used sparingly and that the majority of students accessed less than 10% of the lecture videos (McNulty, et al., 2011;McNulty, Hoyt, et al., 2009).With the exception of anatomy and pharmacology, use of flashcards was another study strategy that received overall low ratings.The higher scores given by students for use of flashcards in anatomy and pharmacology further validates student responses to the survey because these were the only two courses that utilized the online flashcard application developed for the curriculum.It further demonstrates that students will use supplemental resources if they are available.
In conclusion, given that students engage in their study habits prior to the assessment activities that determine the course weighted score, there exists a temporal order of these variables suggesting that the degree to which a student engages in "constructive" study habits may predict their course performance.However we caution against making cause effect statements since this study was not designed to examine the cause and effect relationships between the study approaches and weighted scores.Although study strategies of individual students tend to be consistent from course-to-course, some re-alignments were observed especially in courses that require more memorization (e.g., anatomy and pharmacology).The results could be extended to compare the learning strategies of students in the basic sciences with the clerkships.Other studies could test whether interventions such as increased availability of practice tests influence the study strategies and subsequent performances of individual students.Finally, studies could be designed to examine associations between the use of specific study strategies ("rote" vs. "constructive") and individual learning styles as measured by instruments such as Kohl's Learning Style Inventory (McNulty, Sonntag, & Sinacore, 2009).

Table 1a .
Means for student ratings for study strategies in those courses taken during the first year of basic sciences

Table 1b .
Means for student ratings for study strategies in those courses taken during the second year of basic sciences MHD=Mechanisms of Human Disease; Neuro=Neuroscience; Pharm=Pharmacology and Therapeutics.

Table 2 .
Study methods with loading values of .50 or greater for each of the study approach factors

Table 3 .
Distribution (percent) of students whose study strategies were preferentially "rote", "constructive", or equal between the two The classes are listed chronologically according to the semester in which the classes were taken.Cell=Molecular Biology of the Cell and Genetics; Anat= Human Anatomy; PCM=Patient Centered Medicine; Physio=Physiology; Immuno=Immunology and Microbiology; MHD=Mechanisms of Human Disease; Neuro=Neuroscience; Pharm=Pharmacology and Therapeutics.

Table 4 .
Relationships of study methods versus course performance illustrated by standardized Beta coefficients from regression analysis by course *Significant at p = .05**Significant at p = .01

Table 5 .
Percent of questions that only required knowledge of a fact to answer the question