Designing of Questionnaire for Factors that Impact Student Learning Outcomes in Tertiary Education System: An Analysis from Pakistan

The current research focused on the designing of questionnaire for factors that impact student learning outcomes in tertiary educational system in underdeveloped nation Pakistan. A pilot study was conducted for the designing of questionnaire to collect data on perceived factors that impact student learning outcomes. The selected Higher Education Commission (HEC) recognized private educational institutions were contacted and permission of data collection was obtained from the authorities. Ninety eight students of final semester from HEC recognized tertiary universities at Bachelors level from the department of Science were approached for this study. Participants were informed that the data will be kept in a laptop under password protection. There were 41 total items in the questionnaire. It has been divided into 2 sections. Section A includes the demographic characteristics of participants and includes 7 questions. Section B is divided into 6 subsections, each section capturing the true essence of independent and dependent variables. The results of the study concluded that student perception questionnaire has good to excellent intra rater reliability and indicates that all components have factor loading exceeding 0.50 that also shows the importance of all components. Thus, the questionnaire can be used for the assessment of factors that impact student learning outcomes from educational institutes


Introduction
It is a well recognized fact that economic growth is not possible without expansion and progress of human capital. According to growth theory, education encourages novelty, knowledge of latest technologies and products, thus contribute towards economic growth (Hanushek & Woessmann, 2008). In the Declaration of Human Capital Investment in 1960, Schultz reported that expenditure in education and other related components can hasten the economic growth (Schultz, 1989). The father of economic growth estimated the influence of education to the annual income growth rate and highlighted that education contributed almost thirty five percent in United States. This has concluded that education played as a great contributor to economic prosperity. A positive direct proportional relation between public expenditure on education and economic growth was identified (Easterly & Rebelo, 1993). A study by a group of researchers was conducted to anticipate the role of education spending to economic growth among Asian nations. The findings revealed that contribution of expenditure on education to economic growth constituted almost 10 percent within a period of 10 years. Thus it can be concluded that education is a significant factor that influence economic expansion. Education has always played a significant role in defining human capital.
The main aim of Pakistan government is to provide equal and quality learning opportunities for the citizens, through authentic impovements in education system. Previously, the education system of Pakistan has failed to develop standard learning and quality education. Thus currently, the goal has been set to promote life long learning opportunites and quality education. Table 1 shows the total expenditure on education in Pakistan in terms of current expenditure, development expenditure and as percentage of GDP from year 2013 till 2018 (Kamran, 2018). Source: (Kamran, 2018).
In the year 2017-2018, there were almost 186 universities, fifty six thousand educators and 1.6 million enrolled students in Pakistan. The number of enrolment was increased to almost 7.7 percent than from last years. However in the year 2018-2019, this growth in enrolment was predicted to reduce by 0.2 percent. The public spending expenses on education was projected to be 2.4% of GDP in the year 2017-2018. The government of Pakistan is continually striving to improve financial resources in the education sector. Education expenses have been increasing slowly since 2013-14. The education based expenditure has been raised by 18.6% in year 2017-18. The province wise total expenditure on education has been shown with a graphical representation in Figure 1.0 (Kakar, Khilji, & Khan, 2011). The worth of education is largely dependent on the value of teachers. The main aim of Pakistan government is to prosper the whole world through knowledge. In this situation only an efficient and competent qualified teacher can meet the standards of world class education. The quality of learning depends on quality principles by professional teachers who are enthusiastic and fully devoted (Ahmad, Mahmood, & Ishaq, 2020). The low income country like Pakistan is facing serious challenges in developing quality teachers which consequently leads to poor quality of education, lack of knowledge in studies and weak theoretical background (Khan, Fauzee, & Daud, 2016). Scarce literature is available on the quality of education in Pakistan but a loophole is present while assessing the level of qualified teachers.
It has been practically witnessed that students at university level are not so competitive globally as they memorize and recall knowledge only instead of demonstrating what they have learned. Consequently the graduate students lack practical skills and unable to initiate market oriented work. This problem directs to the factors in education environment which is creating employability gap (Khan & Mohammad, 2018). Prompt attention is needed to evaluate these factors and revise the current practices.
Graduates in Pakistan lack innovation, new discoveries and the businesses here are being functioned as inherited property (Shah & Soomro, 2017). Literature in our society is not sufficient that attempts to comprehend the decreased trend of startups and self-employability. There is window of opportunity for further investigation of ies.ccsenet.org International Education Studies Vol. 16, No. 1;2023 extent of course content related to startups for Pakistani graduates. State and local organizational institute leaders are now shifting their focus towards skill demonstration of their graduates. The method for this paradigm shift is considered as Outcome based education, which defines particular achievement level student, must attain in each domain before graduation.
Several studies have been conducted that determine the impact of OBE in educational organizations and classroom practices especially in the field of engineering. However a literature gap exists when considering the role of different factors perceived by students that impact the knowledge, practical skills, teamwork and employability status. There is an extreme need to assess factors that can enhance learning outcomes of students in the developing nations as human capital largely depends on the education system of the country.

Literature Review
Learning theory defines how the students receive, interpret and retains information during learning process. Many factors play a part in comprehending and retaining the knowledge like cognitive, social, emotional, previous experiences and environmental influences. From the behaviorist perspective learning is a component of conditioning and encourages reward system. The believers of cognitive theory support the description of learning as the complex path of human memory rather than environment. The promoters of constructivism believe that the ability of a learner mostly depends on what they already understand and the new knowledge should be tailored according to the needs of the learner. The theory of transformative learning highlights that continuous change is needed in learner's presumptions (Bransford et al., 2006).
The learning theory that supports this study is Social cognitive theory as it tries to resolve some social ill in Pakistan caused by education. Social cognitive theory included emotions as well as cognitions to the social learning theory. This theory explains behavior of human in terms of three dimensions, dynamic and reciprocal model in which there is interaction of personal, environmental and behavior factors. It generates concepts and processes information from cognitive, behavioristics and emotional models. This theory is often considered as a bridge between the behavioral and cognitive theories as it targets on the collaboration between internal characteristics like thinking, attention, motivation, memory and external factors like rewards and punishments in defining behavior (Schunk & DiBenedetto, 2020).
This theory has its foundation in agentic perspective. Individuals are self-organizing, self-responding, self-regulating, pro-active and not merely reactive to external events (Usher & Schunk, 2018).

Other Models in Education
The other models commonly utilized in education are Inquiry Based Learning (IBL) model, Problem Based Learning (PBL) model and Science, Technology, Engineering and Mathematics (STEM) model. STEM model will provide the theoretical framework for the current study.
STEM model is being discussed here as this model provides emphasis in linking education with practices. It is usual in tertiary education system for students to perform group work activities to solve complex issues in their larger projects. Generally this startegy has been named as problem solving and is often taught in mathematics. However, STEM professionals involve in multifaceted problem solving approach and in majority of cases there are several probable solutions with its strengths and limitations. So, it is vital for students to expand their depth of knowledge that allows them to become successful and reflect on their solutions. STEM strategy develops critical thinkers and highlights that learning and workplace success is interdependent. It focuses that expertise is developed across every discipline even when one has more focus on one suject than the other. Therefore, success at workplace is dependent on knowledge intercation within and across STEM domains. Consequently, the learning environment of students and expectations should replicate this complex learning design. In this model , educator designs expectations for the completion of tasks after identification of learning objectives and incorporate necessary actions (Capraro, Capraro, & Morgan, 2013).

Science , Technology, Engineering and Mathematics (STEM) Model
Countries spend for innovation to promote their economic growth. As several nations are facing challenges from the consequent effects of economic problems for example increasing unemployment and elevating debt, the stability of country and level of development is threatened in the current century. Only innovation generated devolpment has the power to produce value added jobs (Lee, Szapiro, & Mao, 2018). Since innovation is greately generated from progress in STEM field (Council, 2011), a rising number of jobs demand STEM knowledge. Countries require an inventive STEM workforce to show competitive behaviour in this century.
In United States of America (USA) the concept of STEM education has been taken into consideration in 1990s', but a lesser number of educators seemed to understand the application and usage of STEM education even after ies.ccsenet.org International Vol. 16, No. 1;2023 several decades. Americans recognized the fact that the country would stay behind in the worldwide economic development and started to focus on STEM education (Friedman, 2005). The funding for research in STEM domain has been increased potentially in US (Sanders, 2009). The necessity to gain success in Science, Technology, Engineering and Mathematics education is clearly evident by the immense eucational reforms that took place in the previous decades within these disciplines.
Currently, there is a harmony between stakeholders on the significace of STEM education to economic progress (Kuenzi, 2008). STEM education promote interdisciplinary information and skills that are related to real world situations and train students for knowledge driven economy (Council, 2011). The ultimate aim of STEM education is to build up the present generation with innovative minds. STEM education constitutes knowledge, values and skills that are mutually developed at the junction of more than one STEM subject area (Corlu, Capraro, & Capraro, 2014).
It has been shown by the research that STEM approach of teaching is improved when the educator has adequate knowledge of content as well as domain pedagogical content understanding. An integrated strategy seeks to establish associations between STEM disciplines and present a related context for learning. Educators must remain accurate to the nature in which STEM fields are practical to real world conditions (Nadelson, Seifert, Moll, & Coats, 2012).
It has been believed that majority of the content in STEM could be grounded within the theory of Situated Cognition. The core concept of this theory is that comprehending how information, skills and knowledge can be applied is equally important as learning these factors itself. Situated cognition model emphasizes that contexts including both physical and social of learning task are significant to the learning procedure. When the student formulates his knowledge and skill based on an activity, the framework or the environment is important to the learning process. When the process of learning is situated in a context, it becomes valid and relevant, thus seems representative of experience existed in real STEM practice (Putnam & Borko, 2000).
Framework for STEM education shows a Block and tackle of pulleys to lift a weight which is Situated STEM learning. This basically depicts a pulley system that produces mechanical advantage to lift weights easily. The figure shows a linkage between situated learning, engineering design, technological use, scientific research and mathematical thinking as an interconnected system. Each pulley attaches to common practices within STEM disciplines and are bound by the rope of practice. A complex association of pulley system must be synchronized to assure the integrity of whole system (Kelley & Knowles, 2016).

Inquiry Based Model
IBL is a strategy that allows students to involve in the process of knowledge generation and the chief characteristics are learning stimulated through inquiry, an approach towards self-learning, an active process of learning and the student focused procedure. Students must develop skills and become lifelong learners. The foundational aspects of IBL strategy include learning as a result of question formulation or a process of seeking new understanding, a learning method that is students focused in which the responsibility of educator is to provide services of facilitator, self-directed learning in which students taking more responsibility for development of new skills (Spronken-Smith, 2012).

Problem Based Learning (PBL)
In PBL students identify triggers from problem or case scenario to describe their learning objectives. Consequently they perform independent study and become self-directed learners. PBL is not merely the solution of problem; instead it utilizes suitable problems to improve knowledge and understanding. PBL can be considered as a small group learning strategy that incorporates the attainment of knowledge with development of skills. However the introduction of PBL into curriculum also raised some issues in designing and implementing process. Paper based problems form the foundation of core curriculum and ensures that each and every student gets exposed to the similar problem (Word, 2003).

Methods
A pilot study was conducted for the designing of questionnaire to collect data on perceived factors that impact student learning outcomes. The selected Higher Education Commission (HEC) recognized private educational institutions were contacted and permission of data collection was obtained from the authorities. Ninety eight students of final semester from HEC recognized tertiary universities at Bachelors level from the department of Science were approached for this study. The study participants were informed of the procedure before signing the written informed consent. The informed consent will be consisted of the invitation about participation in the study, researcher information, and procedure of the study, purpose of study, potential risk and discomforts of the study, ies.ccsenet.org International Education Studies Vol. 16, No. 1;2023 20 benefits, withdrawal from the study and confidentiality of data. Participants were informed that the data will be kept in a laptop under password protection. There were 41 total items in the questionnaire. It has been divided into 2 sections. Section A includes the demographic characteristics of participants and includes 7 questions (Allen, 2017). Section B is divided into 6 subsections, each section capturing the true essence of independent and dependent variables. Each section has an anchor question which assesses the perception of students regarding factors that will impact student learning outcome. The designing of course content and organization, learning styles and learning methods section has been taken from the work of Higher Education Pakistan in the domain of the Internal Quality Assurance (Commission, 2019). The section of English language competency has been designed from the work conducted in Finland and carried out by a research team in the University of Jyväskylä (Leppänen et al., 2011). The section of level of complexity in subjects has been taken from the concept of Blooms Taxonomy (Bloom, 2013). The section on assessment has been designed from the previous literature focusing the importance of assessment and its impact on student learning outcomes (Jimaa, 2011). The employer graduate requirement section has been taken from a research conducted to identify the key requirements of employers from graduates at the time of recruitment (Pollard et al., 2015). Each subsection has 5 or 6 Likert scale questions that support each variable (DV, IDV and MV). The sections of the questionnaire with number of items have been shown in Table 2  Teaching and learning methods 6 (Commission, 2019) Assessment strategies 8 (Jimaa, 2011) Level of difficulty in courses 5 (Bloom, 2013) Student Preferred learning style 4 (Commission, 2019) English language competency 4 (Leppänen et al., 2011) Employer Graduate requirement 2 (Pollard et al., 2015) Data was analyzed through Statistical Package for Social Sciences (SPSS) version 21. Reliability Analysis and Factor Loading was performed to assess the dimensions of questionnaire. Validity of questionnaire was assessed through Correlation analysis. Data was presented in the tabulated form. For the reliability testing, Cronbach alpha was calculated for each section of the questionnaire. Subsequently, reliability analysis was done by including all the 34 items of the questionnaire of the scale. The value of total Cronbach's alpha was found to be 0.744 that showed good internal consistency. The highest Cronbach alpha was reported to be 0.87 for assessment strategies section of the questionnaire. Each section validity through correlation analysis was done. The value of 0 shows no relationship, values between 0.3 to 0.5 shows moderate relationship and 0.7 shows strong relationship. Note. Extraction method: principal component analysis. Note. Extraction method: principal component analysis; a. 2 components extracted.

Conclusion and Further Research
The main aim of current study was to design a questionnaire and perform a study from tertiary educational institutes in developing nation. The results of the study concluded that student perception questionnaire has good to excellent intra rater reliability and indicates that all components have factor loading exceeding 0.50 that also ies.ccsenet.org International Education Studies Vol. 16, No. 1;2023 22 shows the importance of all components. Thus, the questionnaire can be used for the assessment of factors that impact student learning outcomes from educational institutes.
The strengths of the current study are multifold. It would be beneficial for higher managements of institutes, facilitators, students, parents and the community. On a wider scale it can be helpful for policy makers, designers of curriculum as well as educational departments. Facilitators can recognize the direction they have to select towards modifying the education system and leading towards improvement. This may act as a regulatory means for instructors directing towards application of OBE.
The current study can also be valuable for regulating process whether standard educational set marks are attained and retained at the completion of academic session. Users of education can ascertain if they can accept this novel means of instruction that includes student learning strategy and outcomes are measured at the completion. The study can be useful for the learners as it will facilitate them to learn efficiently keeping in view their learning methods and abilities to aid attainment of projected learning outcome. Since the curriculum is well organized towards student knowledge building, performance level would be more skilled, systematized and proficient.
As the link between education and economic growth of the country has been attested by theories, the resultant of quality education through outcome based education will help in enhancing the economic growth in Pakistan.
The study opened a window of opportunity for future researchers to conduct rigorous studies with strong methodology and large sample size to generalize the findings.