Measuring Student Preferences for Stimulus-Response (Rote) Learning

The study developed and distributed a survey to measure students’ preference for stimulus-response learning. The responses of undergraduate and graduate students suggest the desire to maximize grades fosters a strong preference for instructors who tell students what they need to know and exam questions that incorporate terms and keywords similar to those used in course materials. Although graduate students exhibit a strong partiality for additional elements of stimulus-response learning, they are less likely than undergraduates to prefer courses in which complex assignments are accompanied by step-by-step instructions and most of the required readings are covered by lectures. They also are less prone to focus their exam preparation on items discussed in class. Given the students’ predisposition to replicate information and problem solving strategies conveyed to them, the development of creativity and critical thinking is dependent on students assuming greater responsibility for learning. Instructional strategies for achieving the outcome are discussed.


Problem Statement
Globalization, evolving skill sets, and the accelerating growth of knowledge have significantly increased the contributions of creativity and innovation to economic growth and the necessity of developing critical thinking, creativity, and self-directed, life-long learning skills.Although problem-based, student-directed, and active learning strategies are intended to cultivate these capabilities, the literature indicates there is a mixed record of success (Hung, 2011;Lloyd-Jones & Hak, 2004;Lee, Mann, & Frank, 2010).The outcome is due, in part, to studies focusing on theoretical conceptions and student learning outcomes while giving limited attention to the role of instructional process (Hung, 2011).The discrepancy in coverage has occurred even though there is a preference among faculty and students for stimulus-response learning (Baeten, Dochy, & Struyven, 2008;Gijbels, Segers, & Struyf, 2008;Peters, 2011), i.e., just tell me what I need to know, incorporate into the course assessments the keywords and phrases that will trigger my recollection of the appropriate material, and I will replicate the preferred information and problem solving strategies on the assessments.Due to a preference for stimulus-response learning, efforts to cultivate critical thinking, creativity, and self-directed, life-long learning are undermined.

Definition and Implications of Stimulus-Response Learning
Although the stimulus-response learning and operant conditioning employ similar strategies, there is an important distinction.Instead of relying on a given trigger to yield the desired behavior, stimulus-response learning employs a variety of stimuli to generate the desired response.More specifically, lectures and PowerPoint presentations define the portion of the assigned readings that the instructor deems to be important.The designation is supplemented by exercises such as problem sets, quizzes, study guides, group projects, and case studies that not only reinforce the designation of information that is important and students therefore should know for assessments, but also conveys the acceptable range of problem solving strategies and responses (Peters & Higbea, 2012).By the time students tackle their course assessments, they are conditioned to replicate the strategies and responses rather than searching their knowledge banks for the optimal method of analysis and responses (Boesen, Lithner, & Palm, 2010;Choudhury, Gouldsborough, & Gabriel, 2010;Peters & Higbea, 2012;Uskul & Eaton, 2005).
Due to the instructors' role in defining the relevant information as well as the permissible range of responses and problem solving techniques, they assume primary responsibility for the learning process and thereby enable students to be passive recipients of knowledge and skills.Under these conditions, there is an expectation that instructors will provide sufficient explanation of the material so that it is not necessary for students to independently expend significant amounts of time and energy to develop an understanding of the material.Consequently, there is a tendency to blame the instructor for insufficient understanding of the material, i.e., the instructor has not explained the material in a way that students can understand or to their satisfaction.A student's academic success therefore is perceived as a function of the instructor's teaching skills instead of defining learning as the shared responsibility of instructors and students (Peters & Higbea, 2012).
Students willingly support a system in which faculty define the information and problem solving strategies students should know and replicate for assessments because it enables them to minimize the amount of time and effort they invest in their courses.Students, in more specific terms, are able to forego reading the assigned material with impunity (Brost & Bradley, 2006;Peters & Higbea, 2012).The pervasiveness of this perspective is suggested by the prevalence of aliteracy, i.e., the decision not to read even though one has the ability to do so.Burchfield and Sappington (2000), for example, found that approximately one-third of graduate students and a majority of the undergraduate students choose not to read their assignments.The designation of what students ought to know also enables students to minimize the amount of time and effort they dedicate to comprehension (Sappington, Kinsey, & Munsayac, 2002).They may, in other words, read the text but not invest sufficient time to develop a deep understanding of the material (Boesen, Lithner, & Palm, 2010) and prepare to actively participate in course activities (Peters & Higbea, 2012).A final benefit is minimizing the amount of information students need to study, know, and apply for exams and other assessments (Boesen, Lithner, & Palm, 2010;Brost & Bradley, 2006;Clump & Doll, 2007;Lord, 2008;Ogilvie, 2009;Peters & Higbea, 2012).Since lectures and PowerPoint slides highlight the subset of assigned readings the instructor deems to be important (Adams, 2006;Albers, 2009), and the definition is reinforced by course activities such as case studies, quizzes, and small group projects (Choudhury, Gouldsborough, & Grbriel, 2010;Uskul & Eaton, 2005), students are able to focus on the defined elements and virtually ignore the other material with impunity (Peters & Higbea, 2012).
Given stimulus-response's role in efficiently earning A's and B's, many successful students internalize its rules early in their academic careers.Their allegiance is deepened by the use of grades to track primary school students and for higher education admissions.The educational process, in conjunction with pathways to academic success, therefore nurtures devotees to stimulus-response learning (Albanese, 2000;Albers, 2009;Ogilvie, 2009, Peters & Higbea, 2012;Raidal & Volet, 2009).Due to these elements, college students have a deeply ingrained preference for instructors who differentiate between consequential and inconsequential material (Adams, 2006;Clump & Doll, 2007;Lord, 2008;Peters & Higbea, 2012).

Rationale for Assessing Students' Preference for Stimulus-Response Learning
The statement "darn it professor, just tell me what I need to know for the exam" (Lord, 2008) and related questions such as "what material will be covered on the exam," "what are you (the instructor) looking for," and "what rubric will be used to grade the exercise" are symptomatic of the stimulus-response mindset.Although the strategy maximizes the grades generated by a given amount of studying (Peters, 2011), efficiency is achieved at the expense of fully developing Bloom's higher level cognitive dimensions (Anderson et al., 2001) associated with creativity, critical thinking, and life-long learning.Stimulus-response learning also confounds attempts to distinguish between the extent to which student responses are indicative of learning and understanding or the conditioned response to course-related stimuli (Billing, 2007;Connor-Greene, 2000;Doyle, 1988;Lithner, 2008;Ogilvie, 2009;Peters & Higbea, 2012;Taylor & White, 2006;Watters & Watters, 2007).

Study's Contribution to the Literature
Although a reliance on stimulus-response learning impedes the cultivation of creativity, critical thinking, and life-long learning skills, an assessment of the students' preference for stimulus-response learning has been limited to a few questions that are part of a larger questionnaire measuring students' learning preferences (see, for example, Biggs, 1987).The development and distribution of a stimulus-response questionnaire therefore provides an opportunity to examine the extent to which students enrolled in an undergraduate course and a graduate program are wedded to stimulus-response learning and whether wider distribution of the questionnaire is warranted.More specifically, the questionnaire measures the respondents' propensity to (1) prefer instructors who tell students what they need to know, (2) contend that learning and understanding are a function of the instructor's effectiveness, (3) prefer assessments that focus on information addressed in lectures, case studies, PowerPoint slides, small group activities, and other course-related activities, (4) focus on this set of materials when studying for exams, and (5) choose not to read the assigned materials before they are discussed in class (Peters & Higbea, 2012).
In addition to measuring the students' preference for stimulus-response learning, the study also provides an opportunity to determine whether there is a statistically significant difference in the preferences of males and females as well as undergraduate and graduate students.Given that females are less confident in their test taking abilities and more risk averse than males, this study assesses the presumption that their desire for certainty translates into a stronger preference for stimulus-response learning among women.In addition, both men and women are assumed to move up the ladder of Bloom's cognitive dimensions as they progress from their first year undergraduate courses to their graduate courses.Since progression up the ladder of cognitive dimensions is presumed to be reflected in students who are increasingly independent and self-directed, this study also analyzes the resulting inverse relationship between educational attainment and preference for stimulus-response learning.

Questionnaire Development
A pilot questionnaire was tested in two sections of the Master of Public Administration (MPA) capstone course, revised, and subsequently tested in a MPA health care administration course.Following several additional changes, the questionnaire consists of the 17 questions listed in Table 1.15.
The credential is my primary reason for pursuing a degree.Agree As indicated by Table 1, the survey questions cover the gamut from the assigned readings to exams and the locus of responsibility for understanding the course material.

Participant Selection
The decision to distribute the questionnaire in multiple sections of a 200 level introduction to government course and Master of Public Administration (MPA) courses is based on four factors.First, the undergraduate and graduate courses focus on different levels of Bloom's taxonomy of cognitive dimensions.Due to the 200 level course's focus on the transmission of basic information, it emphasizes the lower level dimensions of remembering and understanding.The MPA courses, on the other hand, endeavor to cultivate the higher level dimensions of analysis, evaluation and creation (Anderson et al., 2001).A comparison of the responses of students enrolled in the two sets of courses therefore provides an opportunity to test the hypothesis that expectations relating to the cognitive dimensions create an environment in which graduate students are more self-directed and therefore less wedded to stimulus-response learning than their undergraduate counterparts.More specifically, the difference of sample means is used to test the following hypotheses.
H 1 : The mean graduate student response for each of the survey questions indicates a lesser allegiance to stimulus-response learning than the mean for the undergraduate student responses.
H 0 : The means for undergraduate and graduate student responses are equal.
Second, although women, on average, earn higher grades than men (Ciani, Summers, & Easter, 2008;Lammers & Riegle-Crumb, 2010), they are more likely to experience test anxiety and are more prone to attribute their academic success to personal effort rather than ability (Harrop, Tattersall, & Goody, 2007).The author's experience in several sections of his undergraduate course suggests these characteristics translate into a desire to minimize risk by expanding the bases on which the course grade is determined.When, in more specific terms, undergraduate students are given the option of increasing their workload by adding an additional component (i.e., concept mapping or summaries of C-SPAN videos) to the course grade formula, a greater proportion of women select the greater workload associated with the additional course component.The primary reason for the decision is a desire to reduce the weight associated with each course component and thereby minimize the potential damage of a bad exam grade.Given the female students' preference for minimizing risk and lesser confidence in their test taking abilities, women are presumed to be more favorable to the increased certainty associated with stimulus-response learning, i.e., highlighting the material students that ought to know for exams, consistency in the wording of exam questions and course materials and the provision of step-by-step instructions for challenging projects.The second hypothesis therefore posits H 2 : The mean response of females to each of the survey questions demonstrates a stronger allegiance to stimulus-response learning than the mean response of their male counterparts.
H 0 : The mean responses for females and male respondents are equal.
Third, given the prevalence of aliteracy, two of the survey questions focus on the proportion of assigned material that students skim and carefully read prior to discussing the information in class.As previously indicated, approximately one-third of graduate students and a majority of the undergraduate students choose not to read their assignments (Burchfield & Sappington, 2000).The third hypothesis therefore posits.
H 3 : The average proportion of the assigned materials that graduate students skim and carefully read before the information is discussed in class is greater than the average proportion read by undergraduates.
H 0 : The average proportion of assigned materials read by undergraduates and graduate respondents is equal.
Fourth, since both sets of courses focus on the development and administration of public programs and draw students from a variety of academic disciplines, less variation in the responses emanates from disciplinary background than would be the case if the two classes were selected from different disciplines or colleges.An associated rationale is the availability of a convenience sample.One of the authors taught a section of the undergraduate course and two of the MPA courses.The connection with both departments and courses facilitated the agreement by multiple professors to distribute the questionnaires in multiple sections/courses and thereby ensure a sample of more than 250 students.

Participant Characteristics
The study's participants were enrolled in an undergraduate introduction to government course and Master of Public Administration (MPA) courses that are offered at a Midwestern university.Instructors for five sections of the undergraduate course and 17 MPA courses were invited to set aside 10 to 15 minutes at the beginning of class for students to complete the questionnaire.Instructors for three of the five sections of the undergraduate class and 14 of the 17 MPA courses agreed to distribute the questionnaire.206 of the 321 students enrolled in the three undergraduate sections and 148 of the 192 students enrolled in the 14 sections of the MPA courses were in attendance and completed the questionnaire on the days that they were distributed in class.

Research Design
To test the hypotheses, a questionnaire containing 15 questions relating to elements of stimulus-response learning and two questions concerning the share of assigned materials read prior to class discussions were distributed to students enrolled in a 200 level introduction to government course and MPA courses.Each survey question is followed by an 11-point Likert-type scale and space for comments.In the case of the 15 questions relating to student preferences for stimulus-response learning, the left and right ends of the scale are labeled strongly disagree and strongly agree, respectively, while the middle (sixth) hatch mark is labeled no opinion.For the two questions relating to the percentage of assigned materials read prior to discussing them in class, the left and right ends of the scale are labeled 0% and 100%, respectively, while the middle hatch mark is labeled 50%.The left end of all scales (i.e., strongly disagree or zero percent of the assigned materials read) is coded 1 and the right end of all scales (i.e., strongly agree or 100 percent of the assigned materials read) is coded 11.
Due to the potential impact of sex (Ciani, Summers, & Easter, 2008;Harrop, Tattersall & Goody, 2007;Lammers, Onwuegbuzie, & Slate, 2007) on the students' perspectives concerning stimulus-response learning, the mean and standard deviation for each question are calculated and reported in Tables 2 and 3 for the following groups: all graduate, all undergraduate, graduate female, graduate male, undergraduate female, and undergraduate male respondents.Questions relating to stimulus-response learning are ranked by the mean scores for all undergraduate respondents.To determine whether there is a statistically significant difference between groups, the difference of means test is calculated and reported in Table 4 for the following combinations of groups (1) all undergraduates and all graduate students, (2) undergraduate females and males, (3) graduate females and males, (4) undergraduate and graduate females, and (5) undergraduate and graduate males.
To develop a deeper understanding of the thinking underlying the responses, a discussion of the means is supplemented by a content analysis of the respondents' comments.More specifically, the comments are categorized by theme.For each of the questions, the most frequent themes and related quotes are reported for those who agree and disagree with the attribute of stimulus-response learning.

Analysis of the Difference of Means
According to the data in Table 2, there is a strong predilection among all categories of respondents for instructors who identify the material students ought to know for the course assessments (survey question 11).The data also indicate that the preference is stronger among undergraduates and women than graduate respondents and men.Although the means for the undergraduate and graduate respondents differ, the foci of their comments are remarkably similar.A majority of both groups contend that telling students what they need to know is helpful and fair.For example, several students stated that "it's nice to know what we should know," "this seems fair and logical," and it is "very helpful," "always beneficial," and "important for me to know expectations of me!" Greater specificity is provided by a minority of the respondents who claim the information enables them to limit the portion of the course materials they study and thereby minimizes the amount of time they allocate to studying.The minority view that rejects this aspect of stimulus-response learning is evident in only one undergraduate and one graduate respondent's comments.Both responses convey the belief that students should be familiar with all of the course material and therefore should not limit themselves to items the instructor defines as important.
The responses relating to survey question 7 (preference for instructors who tell me what I need to know) suggest the identification of important information is not limited to formal communication tools such as study guides.
Due to the acculturation process, students have surmised that information discussed in class and online is deemed to be important by the instructors and therefore is likely to surface on assessment mechanisms such as exams and quizzes.The durability of the perspective is evident in means that are virtually identical for all undergraduate and graduate respondents.However, undergraduate and graduate women exhibit a slightly stronger preference for this mechanism of minimizing risk than their male counterparts.
As is the case for survey question 11, the comments of the undergraduate and graduate respondents are similar.
A majority of the undergraduate and graduate students expressed the belief that if materials are discussed in class, they must be important.The principle is reflected in the comments "I feel the most important topics/ideas are discussed in lecture," "I assume the discussed portions are the most important," and "it is typically inferred that these are the most important readings."Although the responses do not include the probability that the materials will appear on exams, a direct link is provided by a minority of undergraduate and graduate respondents who state "if they (the materials) are both required and discussed then they are more likely to be focal points on exams," "I lean towards material covered in class -knowing that'll be on the exam" and "most professors usually test primarily on what was discussed."However, the perception of a link between the content of class discussions and exams is not universal.Several undergraduate respondents noted that significant portions of their exams focus on materials that are not discussed in class.
For those who reject this element of stimulus-response learning, their perspective is reflected in the comments "I try to focus on all points I think are important" and "[w]e should understand more than what is covered in lectures during class." The desire to maximize the certainty of "what the instructor is looking for" is also reflected in the depth of support for consistency in the wording of exam questions and course materials (survey question 3).Although the means for the survey question exceed 8.5, the evidence indicates that support among the undergraduates is stronger than among the graduate respondents and women are more supportive than men.
The comments of the undergraduate and graduate respondents reveal an implicit understanding of the role of consistent wording in maximizing their course grades.Several undergraduate students observed "if the questions are worded the same then you remember them better and quicker" and "it's easier to recall the info that way."An implicit understanding of the mechanics of stimulus-response learning is also evident in the graduate students' comments "some variation is okay but it should still allow students to remember what the questions refer to specifically from the course," "… in an exam wording helps to recall previous readings" and "it (consistent wording and phrasing) reinforce [s] what you have learned."In each instance, the comments indicate that whenever similar stimuli are used in the course materials and exam questions, the probability of recalling the appropriate response is maximized.There is, however, a price for the maximization of course grades.The tradeoff between responding to the stimuli of similar words and phrases and the development of critical thinking skills is reflected in the observation "that (consistent wording and phrasing) makes it easier to recall answers but doesn't require much critical thinking so it could be bad in some ways." Although most of the remaining comments do not reflect an implicit understanding of stimulus-response learning, they nevertheless reveal a preference for the consistency inherent in this method of learning.Undergraduate and graduate respondents believe consistent wording and phrasing reduces confusion and, in the parlance of undergraduate respondents, reduces or eliminates "tricky" questions.One undergraduate, for example, states "there's no reason to try to throw the students off if you taught them one way, test them that way."An overwhelming share of the comments therefore conveys a belief that consistency is a means of achieving a high degree of certainty regarding an exam question's meaning or intent.Whenever certainty with respect to the assessment questions' meaning or intent is maximized, the respondents' comments suggest students are more confident that they are providing the responses their instructors want and, in the process, are maximizing their exam grades.
However, the preference for the consistency is not universally shared.Approximately three percent (2.9 percent of undergraduate and 3.5 percent of graduate respondents) disagree or strongly disagree with the preference for consistency.The minority sentiment is reflected in the comments "if you know the material it shouldn't matter how it's worded" and "we are to learn concepts and gain knowledge -not regurgitate." The quest to determine "what the instructor is looking for" is also reflected in the students' support for step-by-step instructions when completing challenging projects (survey question 9).As is the case for consistent wording of exam and course materials, the extent of support is greater among undergraduates and female respondents than graduate and male respondents, respectively.
One interpretation of the preference for detailed step-by-step instructions is the respondents' preference for instructors who "hold their hands" and thereby relieve them of the responsibility for independently defining the process by which they will complete an assignment.This perspective is voiced more frequently by undergraduate respondents and is evident in the comments "nice to see an example than (sic) follow the formula," "the teacher should always explain how to accomplish an assignment if they want us to do it right and learn from it," "I don't do good (sic) when I don't know what I'm doing step by step," "not everyone is a good problem solver and can think big assignments through into steps which just leads to doing it wrong, excess stress, and a bad grade," and "it makes completing assignments easier and there is a lot less questions that need to be asked."The perspective is also evident in a graduate respondent's comment "(detailed instructions) help you figure out how to find solutions" while tepid support for hand holding or stimulus-response learning is conveyed by the statement "usually it is easier to see how to get the answer, however, some challenge is beneficial." An alternative perspective is provided by a greater number of the undergraduate and graduate respondents' comments and is more consistent with the views relating to the previously discussed survey questions.Detailed instructions, like the preference for instructors who tell students what they need to know as well as consistent wording and phrasing of course materials and exam questions, minimize the uncertainty of what is expected and thereby maximize the probability of achieving the preferred grade.Several students note "that way you know what the professor wants and you'll get a better grade," "without instructions it is very hard to understand what is expected of us and to meet those expectations," "all instructors have different expectations, therefore I believe they should provide some instruction for what they want," "(instructions should) depend on how they grade.
Instructors are so different in expectations," and "rubrics are very helpful." As is the case for the preceding survey questions, the query relating to detailed instructions generates comments from a minority of students who question the stimulus-response perspective.Several undergraduate respondents suggest that in college there should be an expectation that they independently engage in the problem solving process.A similar view is evident in a graduate respondent's observation that "these are major (graduate) level courses.We should be able to figure it out without the instructor providing step-by-step instruction" while another claims "we should understand and apply key concepts to stretch our understanding of the material and continue to grow."These perspectives, as well as the minority views relating to the previous survey questions, acknowledge that undergraduate and graduate students should be more self-directed and therefore less dependent on stimulus-response learning.However, the data also suggests that the ideal for most students is overwhelmed by the desire to minimize the amount of study time that is necessary to generate the desired grades (Peters & Higbea, 2012).
The preference for certainty and efficiency is also evident in the responses to survey questions 1 and 12.Even though the literature suggests students prefer active learning strategies, such as case studies and small group assignments, more than traditional lectures (McCurry & Martins, 2010;Peters & Higbea, 2012), the data in the two tables suggest the opposite order of affinities.For each of the categories, the mean for lectures (survey question 1) is greater than the mean for discussions (survey question 12).It is also noteworthy that the difference between the means for these two questions is greater for all undergraduates than for all graduate respondents.There also is a difference between the means for undergraduate and graduate women but the difference virtually disappears for the two categories of males.
Although the order of preferences generated by the survey and the literature appear to contradict one another, the conclusion is not supported by a consideration of the literature and survey's foci.The affinity for active learning emanates from studies that analyze learning style preferences while the predilection for lectures is evident when measuring the students' reactions to stimulus-response learning strategies.When viewed from these distinct perspectives, the apparent contradiction between the two sets of conclusions disintegrates.The combination of findings suggest that students prefer instructors who use lectures to efficiently transmit information they define as important and supplement the format with active learning strategies.
The connection between the two apparently disparate conclusions is supported by student responses to the author's open-ended course evaluation question that focuses on the effectiveness of various teaching techniques.Each semester several of the students indicate that the author should reduce the amount of time dedicated to discussion so there is time for lectures that focus on the readings.The evidence therefore suggests that, for a number of students, the desire for lectures that efficiently identify the materials instructors deem to be important trumps the preference for active learning strategies (Peters & Higbea, 2012).
Additional insights into the preference for stimulus-response learning are provided by questions contained in Table 3 that generated undergraduate means between 6.0 and 7.1.When asked whether they prefer detailed PowerPoint presentations (survey question 14), the difference between the means for all undergraduate and all graduate students is primarily due to graduate males exhibiting less of a preference than undergraduate males.However, the similarity of views does not translate into analogous rationales for preferring detailed PowerPoint presentations.The greatest number of undergraduate comments reflect a belief that "the more information the better" while a lesser number state that they prefer detailed PowerPoint presentations if "details need to be known for the exam" or "the instructor is looking for specific detail."Graduate respondents, on the other hand, focus on the framework furnished by the PowerPoint slides.A plurality of the graduate respondent comments contend the outline facilitates the taking of notes while several additional students express a preference for active learning, i.e., class discussions that focus on completing the bare boned framework provided by PowerPoint slides.The latter perspective is evident in the observations "we should be taking an active role in learning" and "I like basic ideas that we expand on orally."In these instances, the instructor assumes the role of facilitator instead of the stimulus-response function of defining and transmitting "important" information.However, the majority sentiments are supplemented by two areas of agreement.A few undergraduate and graduate respondents note that the preferred level of detail is dependent on the material; the more complicated the material, the greater the preferred amount of detail.They also recognize that there are limits to the amount of detail."If there is too much on the PowerPoint I can't tell what is really important to take notes on" and "too much detail is overwhelming." Given the constraints, one of the benefits of detailed presentations is 'having all the correct information to refer to."The assumption concerning the infallibility of PowerPoint presentations is augmented by the supposition that detailed PowerPoint presentations convey information instructors deem to be important and therefore provide a framework around which students can organize their thoughts while studying for exams.Due to these elements, detailed PowerPoint presentations "give you the chance to interact after the PP slides (are presented in class)" and "obviously help with understanding (the material)." Survey questions 10 and 5 measure the extent to which students assign responsibility for insufficient understanding to inadequate preparation (survey question 10) and the quality of instruction (survey question 5).Given stimulus-response learning's underlying premise that instructors assume responsibility for the transmission of knowledge and ultimately the students' success or failure, those who subscribe to the perspective can logically assume it is primarily the instructor's fault when they do not understand the material (Peters & Higbea, 2012).However, a comparison of the means indicates that undergraduate and graduate females are the only groups for which the means for blaming the instructor are greater than the means for inadequate preparation.
The ambivalence regarding the locus of responsibility is also evident in the respondents' comments.An equivalent number of comments focus the blame for insufficient comprehension on instructors and inadequate preparation.The comments also acknowledge the impact of a broader array of factors such as insufficient familiarity with the material, losing focus as lectures progress, the readability of textbooks, insufficient interest in the material covered by the course, and the difficulty of the material.The students' comments therefore suggest that the absence of a definitive locus of responsibility emanates from an appreciation that multiple factors inhibit comprehension.
When asked whether instructors should avoid assigning readings they do not intend to discuss in class (survey question six), the mean for all undergraduates continues to exceed the mean for all graduate respondents.However, the previous order of preferences for women and men is reversed.In this instance, the mean for undergraduate and graduate men is greater than their female counterparts.When examined within the context of undergraduate students' responses to the author's open-ended course evaluation question concerning their decisions to attend or not attend class, the reversal of order is not unexpected.Men, in more specific terms, are more prone to skip class because they feel they are able to glean sufficient information from the textbook to do well on exams.If, given this mindset, instructors add material to the reading list that they do not intend to include in their lectures and therefore are less likely to include on assessments, then the efficiency of reading the text vis-a-vis attending class is diminished.
The role of efficiency, or maximizing the grade generated by a given amount of studying, is also evident in the comments of those who agree and disagree with the survey question.An allegiance to the goals of stimulus-response learning is demonstrated by comments such as "too much reading inhibits focusing on what's important," "(the) instructor is the most important part," and "why have us read and not use the material."The opposing view is evident in the statements "No we are in college [and] we should expect it," "I think a critical part of graduate education is facilitating and encouraging students to be aware of and, hopefully, enter into the 'research conversation'" and "other readings can provide a different understanding of class topics."In these instances, students shoulder the responsibility for understanding the materials not discussed in class as well as defining the connections among and the relevance of all the readings.They are, in other words, more self-directed.
A related extension of the preference for stimulus-response learning is disagreeing with the premise that exam grades should NOT be determined by the replication of material previously covered in the course (survey question eight).In this instance, an allegiance to the goals of stimulus-response learning is demonstrated by lesser means.The data for all respondents and undergraduates therefore maintain the previous patterns of support because the means for all undergraduate respondents and undergraduate women are less than the means for all graduate respondents and undergraduate men, respectively.However, the means for graduate men and women contradict the previous pattern.The mean for graduate men is less than the mean for graduate women.
Most of the undergraduate comments and a majority of the graduate comments support the premise that exams should replicate course material.A number of the comments are based on the previously discussed observation that the material an instructor chooses to discuss in class must be important and therefore should surface on exams.Several undergraduates state "what is the point of a lecture if it's not on the test," "if information on exams aren't conveyed in lectures, handouts or PowerPoints, the whole class period would be irrelevant," and "I prefer those courses because I attend every class and it's pointless if the test is on the book info."Furthermore, the role of replication in stimulus-response learning is evident in the comments "If the information is given to us then it shouldn't be hard to do on exams" and "how else can you grade or measure what's being taught and how much information the students retrieved." Elements of these viewpoints are also evident in graduate respondents' statements such as "I like these courses because it is obvious what to study and I end up remembering more information" and "exams should focus on what is seen in class," However, opposition to stimulus-response learning is evident in one undergraduate and a minority of the graduate respondents' comments.The undergraduate contends "I prefer that my grades depend on critically analyzing the information that has been discussed.I do not like to regurgitate information" while the graduate respondents claim "it is better to discuss critically and develop your own answers," "I prefer independent thinking and analysis, … spewing information presented in class is a waste of time," and "for me, application and advancing questions related to scholarship are what's key." Given the findings of previous studies that graduate students read a greater portion of the assigned materials than undergraduates, the evidence suggests aliteracy (i.e., the ability to read but choosing not to do so), and the associated dependence on instructors to tell them what they ought to know, is more likely to be discarded by graduate students.To test the proposition and compare the results with previous studies, survey questions 16 and 17 asked students to identify the percentage of the required readings that they skim and carefully read, respectively, prior to discussing the materials in class.Due to the use of an 11 point scale where 1 equals zero percent and 11 equals 100 percent, the data indicate that between 30 and 40 percent (means between 4.0 and 5.0) of the undergraduate respondents and approximately 60 percent (mean approximately 7.0) of the graduate students skim or carefully read the assigned materials before they are discussed in class.
Although the undergraduate and graduate means for the two questions differ, the difference is only 0.8 for undergraduates and 0.2 for the graduate respondents.The similarity of means is due, in part, to 37 percent of the undergraduates and 25 percent of the graduate respondents listing identical percentages for both survey questions.When the difference between questions is increased to 20 percentage points (or two points on the 11 point scale), then the percentages for undergraduate and graduate respondents rises to a total of 70 and 64 percent, respectively.When the percentages for each respondent is summed, one-third of the undergraduate respondents and almost half of the graduate respondents skimmed and carefully read more than 120 percent of the required materials prior to discussing them in class.
The respondents' comments suggest the similarity of percentages is a function of the amount of time that is available for reading and the associated decision of whether to skim or carefully read the material.Since the decisions vary throughout the semester and students do not record the care with which they read each assignment, the process of answering the questions apparently is simplified by using as a starting point the average percentage of material they read.The figures are then adjusted as to whether they tend to more frequently skim or carefully read the assignments.
The findings are supported by the literature in two respects.First, the average percentages of materials undergraduate and graduate students read prior to discussing them in class are similar to the results generated by other studies (Burchfield & Sappington, 2000;Clump & Doll, 2007).Second, although other studies do not distinguish between the proportion of assignments that students skim and carefully read, the question was addressed by Sappington, Kinsey and Munsayac (2002).In this instance, students were instructed to carefully read the syllabus before they attended the second class session.Buried within the document was the statement that students could earn extra credit points by sending the instructor an e-mail containing the subject line "bonus."Since only 37 of the 167 students who reported they carefully read the syllabus also complied with the imbedded directive (Sappington, Kinsey & Munsayac, 2002), the finding suggests that students do not distinguish between the two levels of reading.It also lends credence to the similarity of the percentages for materials that are skimmed and carefully read and the sum of the percentages for more than one-third of the respondents exceeding 120 percent.

Analysis of Statistical Significance
The first hypothesis, which is addressed in the participant selection subsection and stipulates a statistically significant (p < .05)difference of means between all undergraduate and graduate respondents, is supported by six of the questions.More specifically, the data in Table 4 indicate there is a statistically significant difference for (1) instructors who tell students what they need to know, (2) consistent wording of exams and course materials, (3) addressing most of the required readings by lecture, (4) exams that are not dependent on reproducing course information, (5) the percentage of the required materials that is skimmed prior to discussing them in class, and ( 6) the percentage of required materials that students carefully read prior to discussing them in class.In each of these instances, there also is a statistically significant difference of means between the responses of undergraduate and graduate women.However, statistically significant differences of means between the responses of undergraduate and graduate men do not apply to (1) addressing most of the required readings by lecture and (2) exams that are not dependent on reproducing course information.Since the first three survey questions listed at the beginning of this paragraph generate means greater than 9.0 for all undergraduate students, the data suggest that the questions generating the strongest support among undergraduates are also the ones that exhibit the greatest deterioration in support among graduate students.a Group variances cannot be assumed to be equal The presumption of the second hypothesis that women are more risk averse and therefore more prone to utilize strategies that maximize the certainty of assessment content and permissible responses is partially supported by the data.Of the 15 questions relating to stimulus-response learning, statistically significant differences of means for undergraduate men and women are produced by (1) consistent wording of exam questions and course material, (2) step-by-step instructions for challenging assignments, and (3) exams that are not based on one's ability to reproduce course information.In the case of graduate students, however, statistically significant difference of means occurs for only one survey question: women are more prone to contend that inadequate understanding of the material s a function of insufficient preparation.The data therefore suggest there are greater similarities than differences in the strength of preferences for strategies that maximize the certainty of information they ought to know for exams and the permissible responses and problem solving strategies.
The hypothesized difference in the proportion of assigned materials that are read by undergraduate and graduate students is supported by the data.There is, in more specific terms, a statistically significant difference in the proportion of the assigned materials read by (1) all undergraduate and graduate respondents, (2) male undergraduates and male graduate respondents, and (3) female undergraduate and female graduate respondents.However, there is not a statistically significant difference between undergraduate females and males or graduate men and women.

Findings
An analysis of the data reveals a strong preference for the efficiencies of stimulus-response learning.The five survey questions generating means in excess of 8.0 on an 11 point scale for both undergraduate and graduate respondents include (1) an affinity for instructors who tell students what they need to know, (2) a preference for exam questions that use terms and keywords that are similar to those employed in course materials such as PowerPoint slides, handouts, and problem sets, (3) a predilection for courses in which most of the required readings are addressed in the lectures, (4) a partiality for complex assignments that include step-by-step instructions rather than expecting students to determine the necessary steps for completing the projects, (5) a propensity to focus on materials that are discussed in class when studying for exams.The message conveyed by the findings is that undergraduate and graduate students express a strong preference for instructors who highlight the subset of materials that students should know for their exams, efficiently convey the important material via lectures, and, in the process, limit the amount of information students need to read and know for exams and other assessments.They also want instructors who maximize the probability of earning high grades by incorporating words and phrases into exam questions that are similar to those used in course materials and provide step-by-step instructions for completing challenging assignments and projects.
There are two equally viable interpretations of the findings.The first asserts students pursue strategies that efficiently use their study time, i.e., maximize the grades generated by a given amount of studying.The desire for efficiency arises from the fact that many, if not most of the undergraduate and graduate students are employed throughout the year.When the amount of time dedicated to work is added to the time commitments for studying, home, family, and community, students have a strong incentive to maximize the efficiency of studying.Each of the preceding five preferences enables students to achieve this goal by minimizing the amount of time dedicated to studying.
The second interpretation, which is closely related to the first, contends that the desired efficiencies are achieved at the expense of promoting deep learning and understanding.From the students' perspective, lectures serve the purpose of designating the portion of assigned readings that the instructor considers to be important as well as the appropriate set of definitions, responses, and problem solving techniques.When lectures are supplemented by problem sets and active learning techniques such as discussions, small group assignments, and case studies, the exercises reinforce and expand upon the appropriate definitions, responses, and problem solving techniques conveyed by the lectures.Due to this process, students are conditioned to provide the appropriate responses and instructors are unable to disentangle the extent to which the students' responses are indicative of an understanding of the material or responding to the stimuli conveyed by various components of the course (Billling, 2007;Connor-Greene, 2000;Doyle, 1988;Lithner, 2008;Ogilvie, 2009;Peters & Higbea, 2012;Taylor & White, 2006;Watters & Watters, 2007).
The difficulty of disentangling the extent to which assessments measure understanding or responses to stimuli is evinced by a section of the author's policy course.During each of the course's class sessions, the discussion focused on the cultural, economic, historical, and political variables that constrain the viable range of policy options.Students were also required to select a policy issue of interest to them and use the course's framework to define the items that affected the viable range of policy options.An assessment of the class discussions and the students' research appeared to indicate that they mastered the application of the framework.However, when the ungraded pre-and post-course questions excluded the course's cues and keywords (i.e., references to cultural, economic, historical, and political variables), less than twenty percent of the students integrated the course's framework into their post-course responses.For the remaining students, the content of the pre-and post-course responses were reminiscent of the responses that would be provided by people on the street who never attended any of the class sessions.
Similar outcomes have been generated each time the author has removed the course's keywords and phrases from a portion of the mid-term or final exam questions.The outcomes of these courses and an analysis of survey responses suggest stimulus-response learning (i.e., tell me the keywords accompanied by the appropriate responses and the relevant problem solving techniques and I will produce the correct answers for the exam or other assessment mechanism) is deeply ingrained in students.The findings are also supported by previous observations that efforts to foster analysis and deep understanding are undermined by the students' desire to efficiently use their study time for the purpose of maximizing grades (Baeten, Dochy, & Struyven, 2008;Gijbels, Segers, & Struyf, 2008).
Even though there is strong support for the preceding elements of stimulus-response learning, a comparison of undergraduate and graduate means reveals a statistically significant reduction in men and women's allegiances to stimulus-response learning as they progress through their undergraduate and graduate degree programs.There are, in more specific terms, statistically significant differences between the responses of undergraduate and graduate men, women, and all respondents for the survey questions (1) instructors who tell students what they need to know, (2) exam grades that are not dependent on reproducing information conveyed by lectures, handouts, and PowerPoint slides (3) the percentage of required readings students skim before they are discussed in class, and (4) the percentage of required readings students carefully read before they are discussed in class.Statistically significant differences of means for all respondents and females are also present for two additional survey questions (1) a predilection for exam words and phrases that are similar to those used in other course material and (2) a preference for courses in which most of the required readings are addressed in lectures.
A further examination of the data suggests the extent of the changes between the undergraduate and graduate student responses are limited by the students' desire for efficiency in generating their preferred grade.Of the five survey questions producing means greater than 8.0, the following three questions do not exhibit statistically significant differences between the mean responses of undergraduate and graduate students (1) a preference for courses in which most of the required readings are covered by lectures, (2) an affinity for step-by-step instructions, and (3) focusing on items discussed in class when studying for exams.A comparison of the survey questions that do and do not generate statistically significant differences suggest graduate students recognize they should engage in the higher level cognitive dimensions of Bloom's taxonomy, but their desire to utilize these skills are constrained by their preference for mechanisms that efficiently transmit information they need to know for assessments and minimize the amount of time they dedicate to studying.

Limitations of Study
Since previous studies relating to learning and instructional preferences have included no or only a few questions relating to student preferences for stimulus-response learning, this study developed and distributed a questionnaire to students enrolled in an undergraduate introduction to government course and MPA courses at a mid-western university.The use of a convenience sample clearly is a major limitation of the study; it findings cannot be generalized to students who are enrolled in other courses at the university much less those taking courses at other universities.However, it does provide insights into student preferences for stimulus-response learning and suggests further refinement and distribution of the questionnaire is warranted.

Implications for Instruction
Although the study's findings cannot be generalized to other undergraduate courses, graduate programs, or universities, the time constraints under which the survey respondents function and the resulting preference for efficiency are not unique.Students across the country endeavor to complete their undergraduate and graduate coursework in a timely manner while they also work and fulfill their social, community, and family obligations.
Due to the numerous demands on their limited time, students naturally gravitate towards the components of stimulus-response learning that enable them to maximize the efficiency of studying, i.e., the return on their investment of time.The desire for efficiency and the resulting preferences for the elements of stimulus-response learning therefore are not unique to the survey respondents.
The evidence also indicates that the broad-based desire for efficiency is achieved at the expense of adequately developing the students' creative and critical thinking skills (Albanese, 2000) that are crucial for life-long learning and their competitiveness in a post-industrial economy.This study's findings therefore suggest that promoting creative and critical thinking skills and a deep rather than a superficial understanding of the material is dependent, to a significant extent, on developing instructional strategies that circumvent the preference for the efficiencies of stimulus-response learning.A key component of the strategies is shifting from instructors to upper-level undergraduate and graduate students a greater share of the responsibility for differentiating between important and unimportant material and their interpretation.The outcomes can be achieved, and students can be induced to read the material, if they are expected to submit concept maps or issues papers before the readings are discussed in class.In both instances, responsibility falls on the students to select the pertinent concepts and either map the way in which they interconnect or to use them to analyze "real world" situations of the students' choosing (Peters & Higbea, 2012).
The preference for stimulus-response learning is further undermined by evaluations/rubrics that are based on the underlying logic of students' work and coverage of the readings rather than a predetermined list of concepts and linkages among them that must be present in the students' submissions.When students perceive that a prescriptive, one size fits all evaluation criteria are not being applied, they increasingly focus on the readings' components that are germane to analyzing and understanding their context at work and less on what they think the instructor may define as important.Under these conditions, there is an additional impetus to develop their creative and reflexive skills and become more reliant on themselves for learning and understanding.
As the learning process moves to the classroom or online, the preference for stimulus-response learning can be further short-circuited by using minimalist, rather than detailed, PowerPoint slides and instructors who assume the role of facilitator rather than director of the learning process.An example of minimalist slides is the six slides that provided the foundation for discussing the evolution of America's healthcare from the mid-1800s to 1960.The slides' text, in their entirety, included (1) mechanisms for producing healthcare, (2) training and credentialing of healthcare providers, (3) stakeholders and their agendas, (4) policy initiatives and variables affecting their implementation, (5) the economy's effect on healthcare financing options, and (6) healthcare financing options and variables affecting their use.When the students are placed in the position of filling in the details of the bare-boned framework and a significant portion of their course grade is determined by their participation in the class discussion, an environment is created in which students assume the lead in making sense of the readings and adding information from their work experience.The author's experience also indicates that the breadth of participation can be maximized when the class size is less than 20 students, class participation grades are posted following each class session, and the grading criterion incorporates a gradual reduction in the grade's increment for each additional time a student contributes to the discussion.The criterion, in other words, provides the greatest incentive (i.e., increase in the class participation grade) to those who are reticent to participate and the least (i.e., minimal or no change for the tenth and subsequent contributions) for those who tend to dominate the conversation.As is the case for concept maps and issue papers, the grading criteria are intended to be an inducement for students to take charge of the learning process and thereby minimize the students' tendency to focus on determining and delivering "what the instructor is looking for."As students transition to taking charge, they are liberated to increasingly engage in creativity and reflexivity.
A final stage in the learning process and opportunity for circumventing the preference for stimulus-response learning is the development of essay questions that not only encourage the utilization of higher level cognitive dimensions such as analysis, evaluation, and creation (Anderson, 2001) but also minimize or eliminate phrases and keywords that were reinforced by several course components, e.g., lectures, case studies, quizzes, and group assignments.As the number phrases and keywords approaches zero, the number of cues regarding the "correct" response also diminishes and students are accorded greater freedom to creatively address the questions.A reduction in phrases and keywords also makes it increasingly feasible to disentangle the students' understanding of the material from their responses to stimuli and thereby gauge the course's impact on students.The flexibility to creatively address the questions also can be enhanced by encouraging students to integrate into their responses information from other courses (Peters, 2014).This approach not only encourages the synthesis of information from a variety of sources but also suggests there is a range of appropriate methods and perspectives for addressing an issue.
Although stimulus-response learning facilitates the efficient use of the students' time, it accentuates their dependence on instructors for cues regarding the permissible range of information and strategies for solving problems.To the extent that instructors adopt strategies that undermine stimulus-response learning, they reward students for cultivating their creative and critical thinking skills.The development of these skills maximizes the probability of producing graduates with enhanced creative, critical thinking, and life-long learning skills.

Table 1 .
Stimulus-response learning survey questions, by category

Table 2 .
Preferences for stimulus-response learning -Mean undergraduate scores greater than 7.2

Table 3 .
Preferences for stimulus-response learning -Mean undergraduate scores less than 7.2

Table 4 .
Survey question p-values