Generic Skills to Reduce Failure Rates in an Undergraduate Accounting Information System Course

This paper reports on an initiative to reduce the failure rate in an undergraduate accounting information systems course without compromising academic standards. The initiative relied on the development of generic skills in the major assessment tasks. The result was to significantly improve the amount of effort students applied to their internal assessment tasks and also to reduce the overall failure rate. Recommendations are made for practice and linked to the literature on failure in academia.


Introduction -the neglected issue of failure within academia
There are surprisingly few publications addressing failure rates within academia.It is an important issue with economic and social consequences for both the failing students and the academic institutions.It is a particularly important issue within Australia because of the decreasing levels of public funding and the heavy dependence on international students to keep the education sector viable (Harman, 2004).
High failure rates deters students from enrolling and reduced enrolments affects the short-term viability of an institution.An obvious temptation is to improve pass rates by lowering standards, but the reputation of the institution would eventually be tarnished and if it became a widespread practice the viability of the whole educational sector would be affected.Within Australia this would be unacceptable because the higher education sector is the third highest export and contributes to around 5% of GDP.
Most academic institutions will try to resist lowering academic standards to maintain the pass rate.However many do not have the option to turn away the lower ability students if they are to remain financially viable.Pedagogical solutions therefore need to be found that reduce failure rates without compromising academic standards.This paper explores this issue through the lens of the high failure rates in an undergraduate accounting information systems course.
1.1 A course with a high failure rate University X is an example of an academic institution having to resolve these tensions.It has over 30,000 students with one third being international students.The business faculty is the largest within the university and more than half of their students are international students.A significant proportion of the business faculty students major in either Accounting or Finance.
Accounting and Information Systems is a compulsory second year undergraduate course in the Accounting major with an unusually high failure rate at University X.This high failure rate has been tolerated because it only applied to this one course and the Accounting Department felt strongly that high standards had to be maintained.Colleagues from other academic institutions have commented with respect on the policy not to pass the weaker students and they commented that other institutions have not resisted the pressure as well as University X.However, the high failure rate sat uncomfortably with the author and provided the motivation to develop interventions to both improve quality and reduce the failure rate.

Hypotheses to explain high failure rates
Informal discussions with staff involved in Accounting Information Systems (AIS) units have identified what they believe are the issues that are leading to the high failure rates.The issues include: international students, late transition, unit content, weak students, poor generic skills, student perception, and low levels of student commitment among failing students.These issues will be used as the frames or lenses through which this study will be analysed.
The paper will explore the issue of academic failure by firstly reviewing the literature for each frame.It will then describe an intervention that was taken to try to reduce failure rates.It will then present the results of the intervention and discuss them in the context of the literature.It will finally conclude with recommendations for further research and practice.

International Students
"Australia has become the third largest exporter of higher education, mainly to a limited number of South East Asian countries" (Harman 2004, p101).International students face several unique difficulties.At one level they may experience more serious alienation than local students because of difficulties in assimilating with university campus life (Klomegah, 2006).The other major difficulty relates to styles of education.
The Chinese and the western styles of education are very different.The Chinese system perceives the teacher as the ultimate authority and discourages disagreement (Ladd and Ruby 1999).The western style accepts and often encourages debate.Huang (2005) reports many variances between Chinese students' perceptions of a good lecture and the way western lectures are conducted.
This creates an uncertain and sometimes stressful environment for international students of South East Asian origin, particularly in subject areas where judgement is required and where there are no predefined right answers.Ladd and Ruby (1999) recommend that instructors investigate international students' learning styles and gradually adjust and introduce western styles to the classroom.Kember (2000) recommends a strategy that focuses on deep learning and group work because Asian students favour collective achievement.He adds that such strategies are also beneficial for a mixed classroom, because they follow good practice guidelines that are of benefit to all students.With group work, Strauss and U. A. (2007) advise that students should be introduced to strategies for dealing with such group conflicts, especially when conflicts could be exacerbated by cultural and linguistic differences.

Late Transition
Many AIS students at University X are late transition students.They complete their first year accounting studies in a college affiliated with the university and enrol in AIS in their first or second semester of joining the university.This imposes a demand on them to adjust to larger class sizes and a higher degree of independent learning.This is a significant challenge because students nowadays are less prepared for university work, and tend to spend less hours studying than their predecessors (Nonis and Hudson, 2006).
During the transition, students may have problems in adopting the cognitive behavioural modifications required at university.They need to "look at themselves as learners, to think about how they learn, to set goals, to actively apply strategies, and to monitor themselves as they advance towards a goal" (Thompson and Geren, 2002, p. 402).

Unit Content
The adjustment is particularly difficult because of the nature of the subject.Accounting Information Systems is a new and developing topic with minimal authoritative guidance from international accounting bodies (Chayeb and Best 2005).It is not a typical accountancy unit, but has a more managerial, decision making, and information systems focus.Within Australia accreditation requirements specify three major topics: Controls, Ethics, Systems Development.Each topic has some key concepts but their application requires considerable judgement.AIS questions seldom have one right answer and students need to interpret ambiguous situations and make and justify recommendations.This clearly sets AIS apart from most accounting and other mathematical units that the students may have completed, as they are often much more structured.
This creates real difficulties for students who expect to rote learn without the intention to understand the underlying context and apply knowledge in different contexts (Kember, 2000).

Weak Students
Most failing students have very low Grade Point Averages (GPAs), indicating a poor performance in their non-AIS units.The intellectual hurdles are compounded by the quality of students undertaking business courses.An anonymous professor lamented that "many have the aspiration but not the ability".
Peer leaning and peer assessment has been recommended and is gaining favour as a learning technique (Topping 2005).Using this approach, students learn from peers by collaboratively studying and assessing other students' work, and evidence suggests that this learning is particularly improved when the assessment method includes feedback about products and processes (Van Den Berg et al, 2006;Bloxham and West, 2004).Benefits include: "Increased student responsibility and autonomy, evaluative skill development, insight into assessment procedures and expectations for high quality work, students work harder with the knowledge that they will be assessed by their peers, potential for providing increased levels of feedback without increased demands on tutors, and encourages deep rather than surface learning."Cassidy (2006, p. 511) Others recommend online facilities to support learning (Kippen, 2003).Flynn, Concannon and Ni Bheachain (2005) argue that such blended learning is of particular importance in accountancy education especially because of the large class sizes.Virtual forums provide opportunities for students to give and receive feedback.

Generic Skills
In addition to any general weaknesses, undergraduate accounting students do not fully understand the importance of generic skills in the accounting profession.They have a tendency to underestimate the importance placed by employers and educators on non-technical skills (such as communication skills) and rank technical accountancy skills much higher (Usoff and Feldmann 1998).
Generic skills include "effective reading, writing, analytical, conceptual thinking and critical thinking skills, an ability to solve problems in an unstructured setting, an understanding of organisations, international and multicultural knowledge and the ability to resolve ethical dilemmas" (Nathan and Dunn, 1997, p189).These skills are almost essential to do well in AIS because a large component of the final exam is usually based on case studies, and students are expected to apply their knowledge of the unit's theoretical concepts.International students face the additional hurdle of having to develop their generic skills in a second language.
Research by Alfan and Othman (2005) on Business and Accounting students in a non English speaking country (Malaysia) found that Chinese students often excel and surpass their peers.This study did not relate to AIS specifically but it suggests that the different styles of teaching and/or language proficiency could be the major barrier faced by similar students in a Western context.
The views on how to rectify this problem range from improving students' generic skills to the introduction of multilingual modules and facilities.This latter suggestion is highly unlikely and impractical in the current context of students with a wide array of diverse linguistic backgrounds but it is mentioned because there is a concern that the current system could lead to discriminatory standards and inherent inequalities (Freidenberg, 2002).A focus on generic skills seems more reasonable and Nathan and Dunn (1997) recommend the use of business press articles to get students to reflect on and debate real life scenarios.When generic skills development involves group work tasks, Woszczynski, Myers, and Moody (2006) advise Information Systems (IS) educators to take into account sensitive ethnic and gender issues, due to the proven differences in values, perceptions, and sensitivities arising from students' cultural backgrounds.

Students' Perception
There is much anecdotal evidence to suggest most students consider AIS to be relatively unimportant.The extent to which this perception is held has never been quantified, but students, especially those with little or no industry experience, find it hard to appreciate the central role of information technology within the accounting profession.
This perceptual problem may arise because "…students' needs and the course's needs sometimes do not match.This can sometimes result in uncertainty and unpredictability for students, which can lead to frustration" (Rodrigues 2004, p. 179)

Students' Commitment
The mistaken student perception manifests itself through poor attendance at lectures, poor preparation for tutorial group discussions, a failure to take advantage of support services, and poor performance with internal assessment tasks.
Students with an Asian background tend to be motivated by their future career focus (Kember, 2000), and their level of commitment to the subject may be adversely affected by their incorrect perception of its importance.
At a more fundamental level, there is strong anecdotal evidence that many students have no interest in accounting what-so-ever and choose to pursue accounting studies due to external factors such as favourable Australian immigration policies towards accountancy graduates (Biggs, 2003).Many students have openly confessed that the only reason they are studying accounting are the favourable immigration policies to meet the high demand for accountants in Australia.
This raises fundamental questions about student motivation and their commitment to study and achieve.The earlier recommendation for group work offers little advice on dealing with group conflicts that arise because of lack of commitment.It was noted earlier, that most students do not take much advantage of the supporting services available to assist them in improving their knowledge and ability in the subject.
Taken as a whole, this consideration of the potential issues suggests that the challenge of reducing the failure rate in AIS is a non-trivial task.

The intervention
The Lecturer-in-Charge of AIS in 2007 semester 1 without the benefit of this literature review, felt that generic skills were the key to improving the situation.He applied for and received a Learning Excellence and Development grant to modify the curriculum to include the development of generic skills.His logic was that students were not adequately prepared during the semester and did badly in the subject as a result.His intervention required students to work both individually and as a group to prepare weekly answers to tutorial exercises and make presentations of their answers during the semester.Generic skills were to be developed through weekly discussions with peers, preparation of the group assignment and presentation of the group answers by individuals.
He believed that students would be inclined to work harder if they had to make a presentation.He constructively aligned the student and course objectives by creating tutorial exercises to match the case study style of exam question in the final exam and in many cases used previous exam questions as the tutorial assignment.He went further and weighted the internal assessment marks to heavily emphasise the group assignments and the individual presentation.A comparison of the changes to the previous semester is tabled below:

Insert table 1 here
In addition to this several new support services were put in place to assist weaker students.These included the provision of Peer Assisted Learning, free voluntary discussion sessions lead by student leaders who have completed the unit with impressive results.Online innovations included a discussion forum and online access to the most recent marks and attendance data.
The normal support services were maintained.These included face-to-face consultation times with lecturers, recording of lectures and making them available online, university study support (how to study, write, reference, and research) and availability of all lecture and tutorial materials online.

Results
The results of the intervention are reported below.Demographic statistics where they are available are reported first to understand the context of the results and confirm earlier hypotheses.
The cohort of AIS students in semester 1 of 2007 was typical of earlier semesters.Table 1 shows that only 14% of the students were local.The remaining 86% were international students with 70% from China or Hong Kong.

Insert table 2 here
Table 2 presents evidence to suggest that the 2007 semester 1 cohort were very weak.At University X students are grouped according to their past results into 5 bands: Band 1 to Band 5.The Band 1 and Band 2 bands represent students that on average have failing grades with Band 1 being the lowest band.There were relatively few students in 2007 semester 1 that had this historical data because the majority were late transition students (see Table 3), but those that did were very weak academically.48% of the students with historical data were in the lowest band and 71% of students had a failing Grade Point Average.

Insert table 3 here
Table 3 summarises how students gained entry into the programme of study.Only those students with a GPA score (6%) were likely to have had prior experience of a large university environment.Only those students with a UAI score (34%) were likely to have had experience of the Australian education system.85% of the students had gained entry to University X through an affiliated college which provided much higher levels of support.The total in table 3 exceeds 100% because students may have multiple qualifications/entry points.

4.1Results of intervention
The internal assessment tasks in 2007 semester 1 were quite different from 2007 semester 2 so the results cannot be directly compared.The 3% improvement in average internal assessment results should therefore not be overemphasised.However, difference in the shape of the distribution curves is very meaningful because both the topics covered and the lecturers of each topic were very similar.
The skew in 2007 semester 2 is -0.38 or close to normal, but the skew in 2007 semester 1 is -1.88 because the majority of students performed very well and left a long thin tail of low performers.It suggests the majority of students worked much harder than in the previous semester and only the very uncommitted students received poor grades in their internal assessment.It is possible the better students will have pulled up the weaker students through the group work, but the group work only accounted for 20% of the internal assessment.The remaining 30% (total of 50%) is the result of individual effort.

Insert figure 1 here
The unscaled raw results of the final exam in 2007 semester 1 are directly comparable to the unscaled raw scores in 2006 semester 2 (Figure 2).The weighting of the topics covered in different sections of the exam case studies were almost identical and the style of question in each section was also the same (e.g.questions mainly based on cases studies to examine interpretative judgement).It is worth noting that the exam structure and nature of questions in the previous semester (2006 semester 1) was also largely the same and there was no unfair advantage for repeating students in the period being studied.
Insert Figure 2 here There was a 3.4% improvement from in the final exam results (Actual averages will not be disclosed to protect privacy).The sample size in 2007-1 was sample size was 507 and the standard deviation was 12.2% (based on students that sat the exam rather than total enrolments).The sample size in 2006-2 was 308 and the standard deviation 12.4%.Assuming a normal t distribution, the probability that the intervention did not made a difference = 0.007% which is highly significant (because it is 4.28 standard deviations away).It is even more significant when we consider that the 2007-1 students appeared to be a weaker cohort than in 2006-2.
It is notable that the improvement was not characterised by a strong skew to the right as with the internal assessment results.This suggests the benefit of working harder for the internal assessment tasks did not carry through in most cases to the final exam.A detailed analysis revealed that the quality of the exam answers was much lower than in the group assignments.It was found that most students answered all the major questions in 2007 semester 1 without scoring zero.This was quite different to 2006-2 where two of the major questions (>10marks) had greater than 30% of students either not answering or scoring zero for the question.In 2007-1 only the last question had higher percentages of students not answering or scoring zero (20%) and many students complained on the online forum that there was not enough time to complete the final exam.The suspicion is that weaker students had been helped by the generic skills assignments to understand the key concepts but did not have the time and sufficient command of English to find the words to adequately express themselves.This suspicion is currently being investigated through a more rigorous quantitative analysis.
Despite these comments, the improvement is still very meaningful.It is a prerequisite of the course to perform satisfactorily in all assessment tasks to pass the course.Students normally perform better in the internal assessment tasks than in the final exam.This means that if they fail, it is usually because they have not performed satisfactorily in their final exam.At University X, individual examiners have the discretion to determine what will be accepted as a satisfactory performance.Figure 3 illustrates several variations.If an examiner had determined that the cut-off for a satisfactory grade was at the first red line in Figure 3, then 8% more students would have passed in 2007-1 than in 2006-2.If an examiner had determined that the cut-off for a satisfactory grade was at the second red line in Figure 3, then 9% more students would have passed in 2007-1 than in 2006-2.If an examiner had determined that the cut-off for a satisfactory grade was at a position between the two red lines in Figure 3, then 6% more students would have passed in 2007-1 than in 2006-2.The reported failure rates are therefore not directly comparable because the examiners have set different cut-off points for satisfactory performance.If they had exactly the same the improvement in failure rates would have been 8%.However because they were not the same it seems more valid to report a 6-9% improvement based on the raw scores.This improvement is noteworthy, but much more improvement is necessary.Ideally the failure rate should be less than 20%, and even stricter criteria for satisfactory performance need to be targeted.

Insert figure 3 here
For completeness the overall results are shown in Figure 3.The overall average improved by 4.2%.However the results are not directly comparable because the weighting for the internal assessment was higher in 2007-1 than in 2006-2.The failure rate was influenced more by the final exam in the majority of cases, and the overall grade is only relevant in determining which passing grade was awarded.
Insert figure 4 here

PAL & other support services
The above discussion strongly suggests the intervention improved the results.We now need to consider whether the improvement was mainly a result of the generic skills intervention or whether the other initiatives such as the introduction of PAL had a significant impact.
It turns out that the majority of students did not attend PAL sessions (309 /524).The 225 students that attended one or more PAL sessions tended to earn better marks than those who did not, particularly as it related to the accounting software assignments (QB1 & QB2) and group assignments.This is surprising because the PAL session did not focus on the accounting software to any degree.
There did not appear to be a correlation between the numbers of PAL sessions attended and the improvement in results although students that attended all 11 PAL sessions had significantly better results than the others.This suggests that PAL attracts the more committed students and their commitment accounts for their improvement in results rather than their attendance at the PAL.

Insert table 5 here
No statistical information was collected to assess student uptake of services to support study.Face-to-face staff consultation times were available to students on every day of the week and students were encouraged to use them to raise issues of concern.The experience of the authors is that few students took advantage of this opportunity.
Students, especially internationals, were also encouraged to take advantage of university support services to improve their writing and other lower level generic skills.However the perception of several teaching staff is that not many students took advantage of these services.Anecdotal evidence suggest that the efforts to promote these services were not effective because many students seemed unaware they were available.This is an area in need of attention because it seems that the weaker students did not gain enough benefit from the group work and need additional assistance to improve their writing skills.

Student Feedback
Student feedback was sought through a standard survey evaluating their experience of the unit.The results are summarised in Table 6 where the score is based on a five point scale corresponding to 5 -strongly agree to 1 -strongly disagree.
The results confirm students clearly understood what was expected of them.Students recognised the learning activities to be both useful and helpful for their understanding and felt that the organisation and teaching staff were of a high quality.It is disturbing that students did not feel the assessment criteria were clearly defined because they were given marking guides every week, and the marking guides were almost exactly what the examiners used to mark the final exam papers in 2006-2.
The main student issues related to workload and the way tutorials were organised.There is no question that there is room for improvement and this will be discussed below.

Discussion
The generic skills intervention and PAL were the two main changes introduced in 2007 semester 1.Most other aspects of the course were unchanged, and the final exam in particular was directly comparable to the previous semester.The result of the interventions was to significantly increase the level of preparation for tutorials during the semester and to reduce the failure rate by 6-9% based on final exam raw scores.It is a particularly significant result because the 2007 semester 1 cohort of students appeared to be weaker than any previous semester.
The evidence suggests the generic skills intervention was responsible for the majority of the improvement.PAL may have contributed in some degree.
The literature suggests why the generic skills intervention was effective.It addressed the issue of student perception by heavily weighting the internal assessment to encourage students to focus on the generic skills assessment tasks.Few students will deliberately set out to fail and it was made clear to them that not only were the generic skills assessment tasks worth 30% of the overall course, they were also strongly aligned to the final exam.Students had a clear understanding that the skills to be mastered in the generic assessment tasks related to 80% of the overall course (internal assessment plus final exam).
The generic skills intervention made extensive use of group work which is recommended in the literature.However the design of the intervention was not informed by the literature and students were not introduced to strategies to deal with group conflicts.This oversight contributed to a number of incidents between students.The issue of free-riding team members was expected but it was not adequately resolved because, on balance, it was decided the pros outweighed the cons in this matter.On reflection one suspects the issue might have been reduced significantly if the issue of group conflicts and their resolution had been introduced during the discussion of ethics, which is a main topic in this unit.

Recommendations for Improvement
The discussion leads to a consideration of what could have been done better.The intervention was successful to a degree, but the failure rate remained unacceptably high.It is suggested that another attempt should be undertaken with an additional lower level generic skills intervention to help the international students with their written English.This however would add even more to the workload and the intervention would have to be redesigned to overcome this issue.
With respect to the workload, it is a big burden to ask students to meet every week (online if necessary) to discuss their answers.The results justify the intervention, but consideration needs to be given to ways of reducing the workload without reducing the benefits.The advantage of reducing the number of group assignments is to also reduce the cost of implementing such an initiative.It was found that the marking of the group assignments took loo long for the individual tutors and caused a great deal of resistance to the initiative.The solution was to relieve tutors of this responsibility by appointing a marker to mark all the group assignments each week.It was fortunate a research grant of $3,000 was available to cover this overhead.It is suggested a slightly smaller but similar sum needs to be available to cover contingencies of this sort for future initiatives.
Finally the tutorials can be improved with a minor change.It seems important to give each student two opportunities to improve their presentation skills but when students are organised into groups of four, it is necessary to have all the students from three different groups present at the beginning of each tutorial.This leaves insufficient time for the students to ask questions and for the tutor to add value.It is a small matter to organise students into groups of five instead of four.If tutorial classes are no more than 25 students, there is no need for more than one group of five students presenting at the start of each tutorial and still give each group a chance to present twice in a semester.
There is also the option of only adding an intervention to improve the lower level generic skills and dropping the group assignments and individual presentation.However, this seems very timid given that the intervention has been tried once with some success.Teachers are encouraged to try variations and report back on their results.The issue seems too important to ignore.Wadsworth, Y. (1997)

Table 5 .
Average Results vs. PAL attendance

Table 6 .
Student Experience of the Unit