Change in Learning and Teaching

Learning and teaching is an important component of university life – critical for students and also for those who teach. Our aim is to make this practice more enjoyable and effective for all involved, and at the same time to encourage a scholarly approach to the process. This volume of Asian Social Science is devoted to the work of a team of academic staff who have collaborated in making real improvements to their teaching and the learning of their students. In this article we will discuss the process of setting up the group, the pedagogical rationale, and the training and support given to all members.

The Division of Economic and Financial Studies (EFS) is the largest Division at Macquarie University with around 4 000 postgraduate students, 9 000 undergraduate students and 300 full and part-time academic staff.The Division offers a mix of disciplines with five departments -Accounting and Finance, Actuarial Studies, Business, Economics, and Statistics.Students can focus on becoming specialist economists, accountants, actuaries, demographers or statisticians or combine other major studies to pursue interests such as organisational psychology, computing, environmental management, languages or other electives.The programs are accredited by the relevant professional bodies.
Faculty members are eager to improve the experiences of their students and, in 2007, the Dean allocated extra funds to assist the development of the learning and teaching process.Several models were discussed and, ultimately, the Learning Excellence and Development (LEAD) team was set up by the Division's Director of Learning and Teaching Studies (Leigh Wood) and the Associate Dean of Learning and Teaching (Peter Petocz), two positions with key responsibility for the learning and teaching process Division-wide.
There are numerous external pressures for academics to demonstrate that their teaching is effective.The Australian Government has set up a Learning and Teaching Performance Fund (LTPF) that is linked to measures of teaching, such as student retention, progression rates through degrees, graduate employment and pay, and students' assessment of their learning at university.University funding is linked to performance on the LTPF.For individual staff members, salary bonuses may be linked to being able to demonstrate the quality of their teaching, so there are extrinsic as well as intrinsic incentives to improve.Of course, the main reason for investigating teaching is to improve the experiences of our students!For a non-Australian audience, let us explain how most teaching is carried out at Macquarie University (and generally in Australian universities).Students enrol in Bachelor degree programs straight from school and most degrees take three years (six semesters), four years for Engineering and six for medicine.Students generally enrol in four subjects per semester, eight per year, 24 for a degree, though there is some variation in the number of credit points per subject.At the end of the three years of study, good students are encouraged to take an honours year and then possibly move into graduate study.
Many subjects will have several large-group lectures per week and associated small-group tutorials.Some of the subjects at Macquarie University, particularly in EFS, have enrolments of over 1 000 students per semester and these are taught in lecture groups of around 500 (this is the size of the largest lecture theatre).The large-group teaching is supported by tutorials or laboratories (depending on the subject) and online learning managers, such as Blackboard.Other ways of learning, such as using peer tutors, are becoming more common (see the article by Dobbie in this volume for an evaluation of the peer assisted learning (PAL) program).
Another influence on teaching is the increased emphasis on generic and professional skills for graduates (see the articles by Evans; Kyng).Teaching is also influenced by the high proportion of international and local Australian students whose first language is not English.These forms a large cohort in undergraduate classes: 67% of the students in the Division of EFS speak a first language that is not English.Ways to use technology in teaching are also critical to student learning (see the articles by Sims; Kyng).

Theoretical approach
Firstly, the present project owes a distant intellectual debt to the 1991 Teaching Enhancement Team, a university-wide pedagogical project to improve the overall quality of learning and teaching at the University of Technology, Sydney, run by their Centre for Learning and Teaching (1992) under the leadership of Professor Ingrid Moses.Both of the authors of this paper were members of that team, and saw how its work re-vitalised the university's approach to learning and teaching matters and helped many academics 'kick start' their pedagogic research.The present LEAD project differs in many ways from the project carried out 16 years earlier, particularly with the targeted professional development and support given.
The theoretical underpinning of the LEAD project is action research methodology (Haggarty & Postlethwaite, 2003).This is any methodology that involves participative researchers who are attempting to make changes to their environment.It involves a cycle of deliberate activity: Planning, Acting, Observing and Reflecting, before reiterating the cycle.Essential elements of these steps are that they are incremental, practicable, comprehensive and reiterated.One of the leading proponents is Jean McNiff (see her online summary, 2002a, or book, 2002b).Action research has developed over the last two decades into an accepted and valid form of enquiry that focuses on a critical approach to practice, undertaken by practitioners within their own practice rather than by researchers outside the practice.McNiff writes: "research is as much about the process of answering questions as it is about the answers themselves" (2002b, p.3).
The participants in the LEAD project were given an overview of action research by Associate Professor Anna Reid from the Macquarie University Learning and Teaching Centre at an early meeting of the team.Not only did we see action research as an appropriate approach to this divisional change and renewal activity, it was also a possible methodology for component projects.Since many of the participants used quantitative methodologies in their discipline research, they were less familiar with an appropriate methodology for investigating their teaching.Therefore we felt that it was important to give them a possible model.
A critical component of any pedagogical change is the evaluation of such change, and we built this aspect into the LEAD program and its individual projects right from the start.Dr Ian Solomonides, also from the University's Learning and Teaching Centre, gave a presentation focusing on evaluation issues at the first full meeting of the team.He referred to the following papers.Alexander (1999) reviewed 104 teaching development projects and he reports that in approximately 90% of cases the project leaders indicated that they had the intention of improving student learning outcomes, but only a third could report this as a concrete outcome since only this third actually evaluated learning outcomes.Peat (2000) questions why we evaluate and identifies four phases of evaluation: formative (is x functional in its context?), summative (is x influencing learning outcomes?),illuminative (how is x really being used by the users?; are there unexpected uses or outcomes?), and integrative (how can x be put to best use in the curriculum and organisation?).
Finally, Olds and Miller (1998) suggested another way of thinking about (project) evaluation by attending to five discrete areas: Table 1.Five areas of evaluation (Olds & Miller, 1998)

Timeline Audience dissemination
What are the project objectives?What questions are you trying to answer?
How will the objectives be met?Which project activities will help you meet each objective?
How will you know the objectives have been met?What measurements will be made?On whom?
When will the measurements be made?
Who needs to know the results?How can you convince them that the objectives were met?

Process
We developed the program to support any staff who wanted to make changes in their curriculum and teaching.Because learning uses many media (online, face-to-face, software, texts and peer learning) we did not restrict the project team to academics.We invited all members of the Division -administrators, technical staff and academics -to apply to join the team.In future iterations of the project, we will examine ways to include students at this planning and selection phase rather than simply as end users of the learning development process.

Selection of team members
The EFS Teaching Excellence Plan was distributed to the Division and staff were invited to join the Learning Excellence and Development (LEAD) team and propose a project.The majority of the projects aligned with the EFS teaching plan -indeed the plan assisted staff to choose a suitable project in their area.We allowed a free choice of projects and no project was rejected, though some were reworked or amalgamated when several people proposed similar plans.
In all, 15 projects were supported, with participants from all academic departments (ranging from early-career lecturers to experienced professors) and including technical and administrative areas.Each team member was supported by a more experienced developer.The projects ranged from an investigation of feedback to students, a web statistics project, mapping of graduate capabilities and the introduction of concordance software for postgraduate students.One project evaluated the peer assistance program run by the Division.A complete list of projects is in Appendix A.

Monthly meetings for professional development
Participants were invited to attend monthly meetings where professional development on relevant topics such as evaluation, action research methodology, project management, ethics and writing pedagogical journal articles was given.
We stressed the importance of evaluating educational change, so evaluation was the first area to be discussed.Meetings were friendly and informal, with refreshments supplied, and most included time for participants to give progress reports and ask questions or get comments from colleagues.Yet the meetings were serious about helping participants define realistic projects, investigate the background, select an appropriate approach and work through the process of implementing changes.

Project planning
We took the simplest approach to project planning by using a GANTT chart (a sample for one of the projects is presented in Appendix B).Participants were all given a chart to adapt for their project.This was the worst-performing aspect of the whole LEAD experience.Participants found it difficult to plan or stick to a plan.There were constantly more pressing problems to deal with: for academics, there will always be pressure on time to review and research, and time management should be taught and practised by all.

Ethics applications
Because the teaching developments were to be evaluated and the results published, human ethics approval was required under Macquarie University and Australian government policy.Further, it was an important goal of the project to discuss ethically appropriate behaviour in pedagogical (and other) research, which usually involves students who are in an unequal power relationship with their lecturers.The Division has an ethics officer, who came along to one of the meetings to answer questions, and the university has an ethics committee that carefully checks all proposals and discusses any potential problems with applicants before giving its approval to the project.
For some of the participants this was the first time they had had to develop an ethics application.Technical research in their discipline often would not require any ethics approval (for example, mathematical research on properties of statistical estimators), but moving to pedagogical research required new skills.This process was therefore a useful way to learn about research in other disciplines -and to add another dimension to their own research.

Peer support
Each participant was allocated an experienced colleague to work with them on their project.Dr Ian Solomonides (from the Learning and Teaching Centre), Peter Petocz and Leigh Wood each worked with several groups (see Appendix A).Even participants who were very experienced in research found that having another colleague regularly phoning or emailing helped with their motivation and progress.Towards the end of 2007 we also formed pairs to review the articles being published in this Journal.

Budget and staffing
Each project was given a budget of $A3 000 (around $US2 500) though some groups also secured extra funding from other sources (such as Project 1, which required the development of a virtual laboratory).The money was available for a range of support, such as research assistance, conference attendance and teaching support (for example, grading of student work), but it was specified that it could not be used for participants' teaching release (we did not want to remove committed teachers from their actual teaching work with students).Many academics in the Division do not normally have access to funds for research assistance and are used to working by themselves.They do not have access to a pool of research assistants (RAs), nor are they familiar with the procedures for employing them.To streamline the process, one of the project co-coordinators, Leigh Wood, employed several research assistants for the overall project, who were then used by individual participants.
The majority of the allocated funding was spent on these RAs to help with the execution and evaluation of projects: they ran focus groups, administered surveys (often for ethical reasons the lecturers could not administer these themselves), performed literature reviews and generally helped with the collection and analysis of data.Eight part-time RAs were used, most of whom were recent graduates.They worked hard and contributed a huge amount to the project.

Withdrawals and obstacles
Not all participants have completed their projects, and some will need several more months to do so.One participant (Project 15) was unable to allocate time to do the project due to staff shortages in her area.Another (Project 3) was promoted and found that he did not have enough time to allocate to the project, though it was completed by a research assistant.Two projects (1 and 8) had technical bugs that are still being sorted out: they are working in prototype but have not been tested with students.One of the projects (Project 2) had problems with ethics approval and so is not reported in this volume.Projects 5 and 13 are proceeding well but not ready for reporting at this stage.

Writing and dissemination
We felt that it was essential that changes in practice should be evaluated and disseminated.This means that we needed to examine the literature around each project and to place our changes in a context.Each participant was supported in writing up their teaching development by a research assistant, who also helped with the literature review; an experienced colleague who reviewed their full paper; and after that a review partner chosen from the team.Finally, the papers in this volume were refereed by a panel of internationally acclaimed experts in learning and teaching.At each stage in the review process, feedback from reviewers was returned to the authors and changes made.Papers were then edited by an experienced journal editor.Not all projects were sufficiently complete to be able to be reported in this volume.
As well as publishing their papers in this volume, participants will be encouraged to share their findings with their colleagues.The first opportunity for all participants to present their work as a group was the LEAD seminar at Macquarie University (in February 2008), though by that time some had already presented their ideas at other prior conferences.Other opportunities will follow at future conferences and in professional journals.

The virtual laboratory
The aim of the project is to allow off-campus access to specialist software through a web browser.Students who require software for their subjects either have to buy a licence for home use or have to come to campus to use the laboratories there.Technical hitches have meant that this project is not complete, though it is working in prototype.

Generic skills in accounting
This project aimed to develop writing skills in a second year accounting Information Technology subject.Many of the students were speakers of languages other than English and it was felt that they needed assistance with their technical writing.Students were given extended weekly writing tasks which were marked and returned quickly.The marking was very onerous for the teaching assistants and the improvement in results for the students was not significant.This teaching intervention seemed not to be successful and is not reported in depth due to problems with ethics clearance.A different approach to developing writing skills needs to be considered.

Mapping graduate capabilities in marketing
In retrospect, this project could have been more ambitious.A research assistant was able to map graduate capabilities in a week and disseminate the results to the department involved.The RA took all the published subject information and created a spreadsheet which clearly showed which capabilities were over-assessed and which were under-assessed.This was done by listing the graduate capabilities required and mapping capabilities assessed in the learning tasks against these capabilities.A more difficult task, which was not done here, would be to validate the graduate capabilities with those actually required by graduates.

Evaluation of peer assisted learning
Peer assisted learning (PAL) has been used in the Division for some years and there is interest in expanding the program to include more subjects.However, PAL is expensive to run, so an evaluation of the current program was undertaken to advise on ways to improve or expand it.The evaluation uses qualitative measures (focus groups of students and peer leaders) to assess the effectiveness of the program.

Training teaching assistants
Examples of subjects from around the globe were collected and a curriculum was designed that would be suitable for the subjects taught by our Division.Due to the long planning cycle for new subjects, this will not be introduced until 2009.

Online quizzes for Operations Research
Many students are enrolled in this subject and the problem was the need for quick feedback to students on test results.
Online tests using randomly generated questions have solved this problem and have resulted in higher pass rates and more positive responses from students.

Student learning approaches
Previous studies have shown that students' learning approaches often become narrower over the course of a university degree.This project aimed to investigate whether this was true for our students and, if so, to propose curriculum strategies to broaden learning approaches.In particular, it was speculated that students whose first language was not English would have a narrower approach in general to learning.This supposition was not found to be true, and the only significant difference was found between undergraduate and postgraduate students.

Developing a statistical package
This project was developed in response to difficulties with teaching large class sizes and the perceived teaching advantage of using individual data for student assignments.The package developed will provide students with an individualised assignment.It is being trialled over the summer courses (January/February in Australia) as there are smaller numbers enrolled in these.During the standard year (in both Semesters 1 and 2) there are usually 1 000 students enrolled, so the risk of trialling software using such a large group was considered to be too high.As this project is not complete the results are not reported in this volume.

A transformed approach to teaching statistics
What can you say about statistics in two minutes?Two hours?Two days?This project has developed and trialled an online textbook for students who are new to statistics.It takes a layered approach to move deeper into statistics theory and its applications, positioning the subject in a way that connects with students' life experiences.

Integrating a sociological approach to accounting
The common perception of accounting is that it is dry and full of numbers.This project looked at the integration of sociological concepts into the study of accounting education, with an emphasis on the themes of social construction and social power.The purpose was to raise student awareness of the nature and functions of accounting in contemporary society.The evaluations were positive.

Reading skills in accounting
This project sought to integrate academic literacy skills into an elective intermediate accounting subject.It was based on setting academic reading and writing tasks through a series of graded published articles, with the aim of preparing students for their senior year and for the workplace.The evaluations were positive.

Spreadsheets in actuarial science
This project also examined graduate capabilities.Students, graduates and employers were surveyed about their use of spreadsheets in actuarial science, and it was found that curriculum changes to include a greater use of spreadsheets would assist graduates as they move into industry.

User uptake of a quick vocabulary tool for graduate students
Many graduate students in Australia have a first language which is not English.This project observed user uptake of concordancing software aimed to assist them with the usage of specialist vocabulary in the writing of their literature reviews.With the small group of graduate students used in this study, the results have been encouraging.The results are too preliminary to be reported here.

Feedback to students
Feedback has been identified as an area where students are often dissatisfied with their university studies.This project used focus groups to find themes, and a survey was designed around these themes to identify students' preferences.The survey was administered to 1 000 students and the results have already been used to refine policy and practice.

Learning in lectures
This project aimed to video-tape small segments of lectures and use these to assist students to learn better in lectures.Due to staff shortages in the area of student support services, the leader of this project was unable to allocate sufficient time to the project and so it was withdrawn.