Investigating Conditions for Student Success at an American University in the Middle East

It is of great significance to an institution of higher education to meet its goals and to establish its institutional effectiveness and that it has a framework for discussing its institutional performance results, accordingly this study aims to investigate a) the conditions for student success at the University with respect to the five benchmarks of effective educational practices (Kuh, 2009); b) significant differences in conditions for student success across important student populations (gender, GPA, number of credits completed, and academic year); and c) how do these conditions contribute to outcomes valued by the institution (students’ growth, satisfaction, and recommendation of the University). Responses of 1853 students’ on the College Outcomes Survey (COS) for years 2007-2010 were used to answer the various research questions of the study. COS Items were selected that measured students’ time allocations and engagement in various activities reflecting effective educational practices, in addition to valued outcomes. Data analysis involved first testing the measurement model and estimating overall fit of the data using confirmatory factor analysis (CFA). Descriptive statistics, and correlations were reported for the benchmarks of effective educational practices, and differences in benchmark experiences by subgroup were investigated. Finally, a structural model tested the influence of benchmarks of academic practice on valued outcomes and a regression was conducted to investigate relationship between student activities and the benchmarks. Results revealed good fit of the data for the model, identified University’s performance on benchmarks of effective educational practices and their relationship to outcomes valued by the University. Implications for practice were discussed.


Introduction
In the last decade, the University, a private 4-year non-profit institution founded in 1866 in Lebanon, has made great progress in realizing its institutional effectiveness.It has been accredited (2004) and then re-accredited (2009), launched its institutional strategic planning process, continuously assesses its processes and programs using an annual assessment plan, and is working on program reviews and outcomes assessment at all levels; course, program and institutional.The above assessments yielded large sets of data.Measureable outputs, quantitative and qualitative, were tracked such as enrollment, retention, graduation, placement, fiscal efficiency, etc.However, the University did not attempt to evaluate the effectiveness of its various approaches or initiatives or to conduct a study that provides evidence of student success or studies the factors that influence it.Not knowing what contributes to exceptional performance makes an institution vulnerable to losing over time what made it successful in the first place (Kuh, Kinzie, Schuh, Whitt, & Associates, 2010).
Changing higher education environment from stagnant college completion rates, gaps in college graduation rates, and external pressures for institutional accountability for student learning have placed strong demand on postsecondary institutions to demonstrate evidence of student success and to better understand the factors that influence it in college.A number of conceptual models of student success and college impact were developed and the following paragraphs will summarize main premises of these models and the related factors.
Many definitions of student success exist, among them are the quantifiable student attainment indicators such as grades, persistence, length of time to degree, degree attainment, graduate school admission test scores, etc (Venezia, Kirst, & Antonio, 2003).Kuh, Kinzie, Buckley, Bridges and Hayek (2007) proposed a broad definition of student success to include academic achievement, engagement in educationally purposeful activities, satisfaction, acquisition of desired knowledge, skills and competencies, persistence, attainment of educational objectives, and post college performance.
Models that examine student success include five sets of variables: (1) student background characteristics including demographics and pre-college academic and other experiences, (2) structural characteristics of institutions, (3) interactions with faculty and staff members and peers, (4) student perceptions of the learning environment, and (5) the quality of effort students devote to educationally purposeful activities (Kuh et al., 2007).With respect to the above, research findings indicate that pre-college characteristics do not explain everything that matters to student success in college (Astin, 1993;Pace, 1990;Pascarella & Terenzini, 2005).Once college experiences are taken into account-living on campus, enrollment status, working off campus and so forth-the effects of pre-college characteristics and experiences diminish considerably.Student engagement positively affects grades in both the first and last year of college as well as persistence to the second year at the same institution, even after controlling for a host of pre-college characteristics and other variables linked with these outcomes, such as merit aid and parental education (Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008).
Accordingly and after controlling for precollege experiences, it is evident that student success is highly related to the college experience itself which includes two dimensions: institutional conditions, effort institutions devote to using effective educational practices, and student behaviors, time and energy students invest in educationally purposeful activities.
With respect to institutional effort, Chickering and Gamson (1987) synthesized the existing evidence on the impact of college on students and categorized it into seven broad principles for good practice in undergraduate education.These seven principles are: (a) encouraging student-faculty contact, (b) encouraging cooperation among students, (c) encouraging active learning, (d) giving prompt feedback to students, (e) emphasizing time on task, (f) communicating high expectations, and (g) respecting diverse talents and ways of learning.A large body of evidence exists to support the predictive validity of Chickering and Gamson's (1987) principles for good practice in undergraduate education.Various measures of these principles for good practice are significantly and positively linked to desired aspects of cognitive and non-cognitive growth during college, and career and personal benefits after college (Cruce, Wolniak, Seifert, & Pascarella, 2006).Kuh et al. (2008) calls some undergraduate opportunities provided by some institutions, such as learning communities, service-learning, research with a faculty member, study abroad, internships, and culminating senior experiences, "high impact practices" because of their positive effects on student learning and retention.Participation in these practices can be life-changing (NSSE, 2010).
With respect to students, what students do during college counts more in terms of what they learn and whether they will persist in college than who they are or even where they go to college.That is, the extensive research on college student development shows that the time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development (Astin, 1993;Pace, 1980;Pascarella & Terenzini, 1991).Studies show that engagement is positively related to test scores and students' reports of learning (Gellin, 2003;Kuh, Hu, & Vesper, 2000;Pike, Kuh, & Gonyea, 2003) and that institutional actions influence levels of engagement and learning on campus (Kuh, Schuh, Whitt, & Associates, 1991).Moreover, different types of engagement have been found to be differentially related to learning outcomes (Pike, 2006).
To provide a common language and framework for discussing and reporting student engagement results, National Survey of Student Engagement (NSSE) used a combination of empirical and conceptual analyses to identify a small number of clusters, or benchmarks, of effective educational practice.The benchmarks are based on forty-two key questions from the NSSE that capture many of the most important aspects of the student experience.These student behaviors and institutional features are some of the more powerful contributors to learning and personal development.They include (a) Level of Academic Challenge, (b) Active and Collaborative Learning, (c) Student Faculty Interaction, (d) Enriching Educational Experiences, and (e) Supporting Campus Environment.
Institutional impact is not just a function of human and financial resources and prestige, but rather the result of purposeful action.Many of the dimensions of good practices are amenable to purposeful intervention or thoughtful planning (Cruce et al., 2006).All students attending institutions that employ a comprehensive system of complementary initiatives based on effective educational practices are more likely to perform better academically, to be more satisfied, and to persist and graduate (Kuh, Kinzie, Schuh, Whitt, & Associates, 2005;Kuh et al., 2007), especially those who start college with two or more "risk" factors (Cruce et al., 2006;Kuh et al., 2008).Therefore, institutions should try to direct student energy toward these educationally effective activities.The University needs to study how well it is providing the conditions that positively affect student knowledge base and foster student success thus realizing an important element of institutional effectiveness.It needs to identify what it is not doing that it should, and what are the policies and practices that need to be adapted to its unique context and circumstances to better realize its mission.
Accordingly and as it is of great significance to the University that its institutional effectiveness in meeting its goals is established, and that there is a framework and a model for discussing its institutional performance results, this research undertakes to study the following: 1) What are the conditions for student success at the University in terms of the five benchmarks of effective educational practices (Kuh, 2009).
2) Are there significant differences in conditions for student success across important student populations?
3) How do the conditions for student success contribute to valued outcomes?
It is important for the University to generate a valid model for the success of its students and for identifying benchmarks of effective educational practices that provides a good fit of its institutional data.Such a model would help identify what is missing and how situation can be improved with appropriate teaching practices and programmatic interventions.Kane (2006) reports that validation helps to establish the legitimacy of a model and accordingly supports inferences and arguments based on its results.Identifying overall conditions of student success might not be sufficient, as they could differ by students' characteristics.Research reports that though exposure to effective educational practices benefits all students, yet it has a compensatory effect in that its effects are greater for lower ability students (Kuh et al., 2008).Similarly, effects are different for freshmen and seniors.So it is important to learn which educational practices work best under what circumstances for different types of students in order to improve the quality of student experience and enhance chances of success.Finally, identifying the complex relationship between educational practices, student behaviors, and outcomes is highly needed, as relatively few studies have explored it (Gordon, Ludlum, & Hoey, 2008) and question was not investigated before at the University.In addition and as the University operates in a different cultural and religious context, it is important to investigate the generalizability of student success models and their predictors in a different culture.There is a need for such a research as a greater number of institutions of higher education in the region are adopting and following the American model and its practices and it is important to assess the applicability of these models to local cultural context.

Research Model
The research model builds on Kuh's (2009) five benchmarks of academic practice.Student activities (course related, learning experiences, employment and social activities), are hypothesized to promote student experiences along the benchmarks of academic practice.Engaging in activities involving these benchmarks not only contributes to student learning and personal development (growth) but also improves valued outcomes at the University (student satisfaction, recommendation of the University).Figure 1 provides a representation of the model.

Procedure and Measure
The University periodically collects information from its various stakeholders as part of its annual assessment plan.One of the tools used is the ACT College Outcomes Survey (COS) and it is used for assessing student satisfaction with the institution and the degree of attainment of various learning outcomes.The COS is administered annually in spring to a representative sample of undergraduate students.Its various sections provide information on student growth in various college outcomes, in addition to information on student achievements and goals, responsibilities and time allocations, and their satisfaction with various aspects of the university.This research uses data base of 1853 students' responses on the COS for years 2007-2010 to answer the various research questions of the study.Items were selected from the COS that measured students' time allocations and engagement in various activities reflecting benchmarks of effective educational practices, in addition to valued outcomes like students' growth, satisfaction, and their recommendation of the University.Such an approach of developing institutional scalelets was recommended by Pike (2006) for institutions not participating in NSSE but is using other instruments and who wanted to document impact on students' development and learning.Appendix provides list of COS items used to answer research questions.

Variables
The dependent variables for the study are the five measures of students' growth in various domains (intellectual, social, personal, preparation for graduate work, and preparation for a career) and three measures of word of mouth recommendation of the University and a measure of student satisfaction with college experience.Student activities (course related, other learning experiences, social, employment) that promoted student engagement in the five benchmarks of effective educational practice constitute the independent variables of the study.The construct validity of the measures was investigated with CFA and the results (Table 1) provide evidence of their reliability and convergent validity.Kuh's (2009) benchmark definitions, items that tap the content of the benchmark were assigned to each benchmark.Then confirmatory factor analysis (CFA) was conducted by benchmark to test for convergent validity and analyze the dimensionality of each measure.This was followed by model specification and the overall measurement model including all benchmark dimensions was used to test for discriminant validity and estimate the overall fit to the data.Descriptive statistics, percentiles, and correlations were reported for the benchmarks and differences in benchmark experiences were investigated pair wise by gender, GPA, number of credits completed, and academic year.Finally, a structural model tested the influence of benchmarks of academic practice on valued outcomes and a regression was conducted to investigate relationship between student activities and the benchmarks.

Results
Based on items included in benchmarks, results of CFA for each of the benchmarks indicated a second-order factor structure for the five benchmarks of academic practice.Level of academic challenge revealed two sub-dimensions of challenge in terms of workload and content, while active and collaborative learning loaded on the two components of internal and external collaboration.Similarly, supportive campus environment included satisfaction with campus relationships and with services.The simultaneous measurement model of all benchmarks and the second-order factor structure fits the University data well (Table 2).Fit indices report a good fit of the model in relation to the data of RMSEA= 0.04, CFI=0.93, and NFI=0.90 are close to cutoff values and supportive of a good fit in relation to the data.Details of the benchmark measures are reported in Table 3 with their path and reliability coefficients.Items with high cross-loadings or non-significant path coefficients were dropped.As revealed in (Table 3), all measures and sub-dimensions had good reliabilities ≥ 0.70, with the exception of external collaboration with internal consistency of 0.56.Also, path coefficients were high ≥ 0.70 and all items loaded on their measures suggesting that most of the variance has been explained and affirming convergent validity of the model.Kline (2005) defines convergent validity as items measuring same construct and correlating strongly among themselves, while displaying low correlations with items indexing different constructs (discriminant validity).Inter-correlations among the benchmarks are low to moderate ranging between 0.21-0.64indicating relatively independent factors and good discriminant validity.Similarly, correlations between benchmarks activities and valued outcomes are all significant and moderate with the exception of correlations with student faculty interaction which are low (R=0.20).Table also reports correlations between student activities and benchmarks and it is evident that social activities have a significant but low relationship to nearly all benchmarks, and that learning experiences outside the classroom contribute to providing academic challenge and to student faculty interaction.
Differences in benchmark experiences were investigated pair wise and only showed difference by academic year and not by GPA, gender, or number of credit hours completed.University's averages in 2007 on academic challenge, enriching educational experiences and supportive campus climate were significantly higher than subsequent years.This could be due to the fact that the University in 2007 was actively working on its self-study to obtain re-accreditation and great emphasis was placed on these effective educational practices.Similarly, level of academic challenge showed significant difference by number of completed credits in favor of those with more than 43 credits.Students with more advanced standing faced higher level of academic challenge than those with lower standing.
Regression of benchmarks on student activities (Table 5) confirmed influence of university social activities on active & collaborative learning, providing an enriching educational experience and level of academic challenge.Of interest, is that off-campus employment negatively affects active & collaborative learning while on-campus one enhances it.Results of overall structural model show that model fits University data well and that an enriching educational experience and supportive campus climate improve all outcomes.The level of academic challenge leads to student growth and positive recommendations, while student faculty interaction negatively contributed to word of mouth recommendation and to satisfaction, and active and collaborative learning negatively contributed to overall student satisfaction (Table 5).Examining structural model by year revealed that an enriching educational experience and supportive environment supported student growth in all three years.University's reputation is built on the level of academic challenge and supportive campus environment, while student satisfaction is built on supportive campus environment (Table 6).

Discussion
The purpose of this study was to identify conditions for student success at the University.The measurement model on which investigation was done was first investigated.Results provided valid and reliable measures and fit of the University data to the model, affirming the convergent validity of the model.The low inter-correlations among the benchmarks also confirmed independence of the factors and good discriminant validity.The relationship between time allotted by students to various activities and their engagement in effective educational practices was identified, and finally, relationship between this engagement and valued outcomes was investigated.
Results revealed that student engagement in educationally purposeful activities is positively related to valued outcomes.Student engagement in on-campus social activities, course related, and other learning experiences at the University contributed to student experiencing of effective educational practices.On-campus work employment also contributes to providing effective educational experience but not off-campus employment.
Various models of student success in college highlight the importance of academic and social integration (Astin, 1993)  Extracurricular activities that facilitate social integration are expected to positively relate to student outcomes, while those that pull students away from campus are expected to negatively affect these outcomes.With respect to work experience, on-campus employment is associated with more positive outcomes, while off-campus employment is seen to inhibit students' integration and involvement (Arum & Roksa, 2011).Hanson, Weeden & Valiga, (2011) summarizing four years of college outcomes in Wabash Study reported that working on and off-campus up to 20 hours during first year of college has little impact on students' cognitive development, however, students working off-campus for more than 20 hours per week were disadvantaged, when compared to their peers, slightly more than 0.25 of a standard deviation in critical thinking skills.According to Arum & Roksa, previous research was more consistent regarding the negative effective of off-campus employment; however, more recent research (Pascarella & Terenzini, 2005) produced a more mixed set of results.
Students' experiencing a supportive campus climate (campus activities, informal contact and availability of faculty, etc) and an enriching educational experience (acquiring general education, academic competency, social skills, etc) contributed most to their growth, satisfaction with University, and their word of mouth recommendation of it (Table 5).Students' experiencing academic challenge (given opportunity to develop thinking, writing, application skills, etc) followed next as a contributor to students' growth and word of mouth recommendation.This confirms research findings of Pike (2006) that different types of student engagement have been found to be differently related to learning outcomes.For example, greater involvement with writing was positively related to gains in general education and in writing, and experience with information technology was positively related to gains in practical skills.Also, this study's findings confirm results of a study investigating effectiveness of NSSE benchmarks.Pascarella, Seifert and Blaich (2010) concluded that across all liberal arts outcomes, the most influential NSSE benchmarks appeared to be enriching educational experiences, followed by supportive campus environment and academic challenge.Only student faculty interaction benchmark failed to have a significant partial correlation with any of the seven liberal arts outcomes investigated, which has also been confirmed in this study.
According to the descriptive statistics, the University is doing better on providing EEE and AC i.e. it is focusing more on academic side but needs to work on improving its SCC through creating the supportive social psychological environment in terms of appropriate interconnected learning support networks and programmatic interventions.According to Kuh et. Al. (2010), student success is enhanced when an institution provides many complementary policies and practices to support students academically and socially.The University also needs to work on improving its performance on student faculty interaction (SFI) and on active and collaborative learning (ACL) and investigate the negative effect they are now having on student satisfaction.Gordon et al. (2008) reported a negative correlation between faculty student relationships and senior students with higher GPAs.This finding is also supported by recent research (Arum & Roksa, 2011), where a negative relationship was reported between learning and studying with peers or engaging with them in different activities (clubs, fraternities, etc.).
Differences in students' experiencing of effective educational practices were only noted by year, and by class level for academic challenge, and not by GPA or gender.Kuh, et. al. (2005) report that all students attending institutions that employ a system of initiatives based on effective educational practices are more likely to perform better academically and to be more satisfied.University needs to regain its 2007 activity on educationally effective practices and improve it, and at same time to work on lower level students to enhance their academic challenge.Some initiatives can be introduced like well-designed orientation, first-year seminars, learning communities, relevant advising, supplemental instruction, etc. as these have demonstrated effectiveness in enhancing student success (Kuh et al., 2008).
Finally, results confirmed the applicability of the measurement model on student success and its predictors in a different cultural context.Each institution has its own cultural traditions and distinctive features, and it is important to identify the combination of policies, programs, and practices that foster student success and ensure institution's educational effectiveness.Having a conceptual model, allows greater generalizability with respect to understanding the impact of effective educational practices.
There are several limitations to this study.First, it is limited to the study of a single institution and therefore the generalizability of the findings is limited to students of similar institutions.Also, study uses self-report gains and although self-report data have been extensively studied and yield valid information (Kuh, 2001), yet self-report gains do not provide stringent controls for the development of students.
This study has validated a model of student success at the University and has provided information on the extent to which conditions for student success exist and areas that need improvement.To create a culture that promotes student achievement and success, it is recommended that the systematic auditing of these conditions be institutionalized.Future research should investigate reasons behind negative relationship between some benchmarks and outcomes, and study the effect of various interventions on students with different characteristics.

Figure
Figure 1.Measurement model

Table 1 .
Path and reliability coefficients of measures Data analysis involved first testing the measurement model.Item selection was done based on a combination of content and empirical analyses.Based on

Table 2
Table4reports descriptive statistics and correlations and results show link between student activities and valued outcomes.More evident from the benchmark means, the University is doing significantly better on enriching educational experiences and level of academic challenge than on other educational practices.

Table 5 .
Model Results by Benchmarks.Significant βs

Table 6 .
Model results by year, significant βs.