A Digital Ecosystems Model of Assessment Feedback on Student Learning

The term ecosystem has been used to describe complex interactions between living organisms and the physical world. The principles underlying ecosystems can also be applied to complex human interactions in the digital world. As internet technologies make an increasing contribution to teaching and learning practice in higher education, the principles of digital ecosystems may help us understand how to maximise technology to benefit active, self-regulated learning especially among groups of learners. Here, feedback on student learning is presented within a conceptual digital ecosystems model of learning. Additionally, we have developed a Web 2.0-based system, called ASSET, which incorporates multimedia and social networking features to deliver assessment feedback within the functionality of the digital ecosystems model. Both the digital ecosystems model and the ASSET system are described and their implications for enhancing feedback on student learning are discussed.


Introduction
The term ecosystem originated from the work of Tansley and Clapham early last century to describe an ecological community of organisms (i.e., plants and animals) living within and sharing a physical environment (Wills, 1997).
Ecosystems comprise a complex network of interactions and energy flows that occur between organisms and between organisms and their physical environment.Because ecosystems are dynamically evolving entities, they are subject to periodic disturbances and consequently are in a regular state of flux.
The principles of ecosystems also describe activities involving complex interactions between people (Briscoe, 2009).For instance, the characteristics of biological ecosystems have been modelled in software programs to simulate socio-economic growth in e-business contexts (Dini et al., 2005) and for the entertainment industry (Bennett, 2006) and the principles of ecosystems are well suited to describe online social networking as this popular activity fosters complex, self-organising interactions among people, and between people and internet technology.
Like any biologically-based ecosystem, its digital web correlate has itself evolved to become Web 2.0.Where Web 1.0 provided a more top-down, hierarchical approach for delivery of content from author to user, the Web 2.0 upgrade puts greater emphasis on: user-generated content, data and content sharing, and collaboration.These features allow new ways of interacting with web-based applications so that the web can be used as a platform for generating, re-purposing and consuming content (Franklin & van Harmelen, 2007).Web 2.0 is now so pervasive on the internet that it has become the de facto state of the web, spawning the term 'social web'.
The social aspects of Web 2.0 have allowed websites such as YouTube, Facebook, Twitter and Wikipedia to dominate the online world.These, and similar websites, share characteristics of digital ecosystems in that they: involve complex interactions between self-organising communities; are essentially decentralised in terms of content production; are self-sustaining and populated with user-generated content; and allow dynamic interactions between people and technology.Social media and networking websites are both reactive to change as well as effecting change, as was evidenced by the influence of Twitter during the Arab Spring of 2011 (Stepanova, 2011).
There are opportunity gains to applying the concepts of digital ecosystems to educational contexts (Reyna, 2011) and numerous studies have been made into the use of popular social networking sites to support formal learning in higher education (Novak et al., 2012).We wanted to explore one particular aspect of the learning process, that of feedback on student learning, and see if it could be modelled and supported within a digital ecosystems learning framework.
The terms assessment and feedback encompass a wide variety of theories and practices.In a review of assessment theories, Taras (2012) paid particular attention to 'assessment for learning'.In terms of feedback, there is much interest in students' comprehension and use of feedback, especially for feeding forward to future learning (Sadler, 1989;QAA, 2006;Weaver, 2006;Murtagh & Baker, 2009).We have previously reviewed the literature around assessment feedback and highlighted the importance of feedback as a participatory dialogic process between tutors and students (Orsmond et al., 2011).Current feedback practices do not always offer adequate opportunities for on-going dialogue, making learning less effective and contributing to low student satisfaction with feedback (Bloxham & Campbell, 2010).Universities have attempted to improve student satisfaction levels through policies to provide more rapid and higher quality feedback, but Nicol (2010) suggests this cannot be achieved solely by increasing the amount of feedback provided, as students also seek opportunities to discuss their work.However, the task of supporting tutor-student dialogues around each assessment using traditional, face-to-face, one-to-one approaches can be prohibitively resource-intensive (Bloxham & Campbell, 2010), particularly in terms of increased workload demands on tutors.
Social networking sites may offer low-cost yet effective opportunities for engaging students and tutors in on-going dialogue.As social media sites are built around a participatory approach, they allow greater collaboration through sharing and creation of knowledge in web communities.Combining interaction with multimedia has numerous benefits as studies have shown that video-based communication made tutors appear more real, present, and familiar to students in ways similar to face-to-face instruction (Borup et al., 2012) thereby offering a far more engaging process than just reading tutors' written comments.
As the power and influence of internet technologies continue to grow, so does the literature base supporting the use of social media in education and in supporting the transition of students to higher education (DeAndrea et al., 2012).Indeed, many universities already use social sites such as Facebook, YouTube and Twitter to support and communicate with students pre-and post-entry.However, Brown (2012) found, perhaps not surprisingly, that academics felt the blanket use of Web 2.0 to promote student learning may not be appropriate at all times and in all contexts.While the use of social networking sites in higher education offers numerous benefits, there are issues around using students' social spaces by academics (Grosseck, 2009) and loss of control of, sometimes sensitive and personal, university-related material when uploaded to public spaces.
As an alternative approach to using existing, open social websites, we developed a bespoke, online resource that brought together the technologies underlying the social web but applied in an academic context, specifically to support tutor-student and student-student dialogue within a digital ecosystems framework to focus on assessment feedback.The system was called ASSET (ASSessment Enhancement Tool), and Crook et al., (2011) reported on its use with 27 staff and 297 students across all disciplines at Reading University and found highly positive student and tutor attitudes to using video-based assessment feedback.Here, we discuss how we have used a digital ecosystems approach to conceptually model feedback on student learning and the subsequent development of ASSET as a web-based system that can support a participatory and dialogic learning process.

A Digital Ecosystems Model of Teaching and Learning
Biological ecosystems are divided into the biotic or 'live' matter component (i.e., living organisms) and abiotic or 'non-living ', physical component (e.g., air, soil, water, sunlight, minerals etc.).Reyna (2011) proposed a digital teaching and learning ecosystem (DTLE) model in which the 'biotic' components comprised the 'teaching niche' (i.e., lecturers, tutors, learning technologists) and 'learning niche' (i.e., students).The 'abiotic' component comprised the physical devices used to access content (e.g., computers, laptops, tablets etc.), the internet connection (e.g., broadband, 3G, 4G etc.), the e-learning interface and the content, either static (i.e., resources loaded onto the e-learning site) or dynamic (i.e., communication tools, collaborative tools and assessments).The model also lends itself to digital correlates of biological aspects such as biodiversity, symbiotic relationships, conservation, balance and adaptation (Reyna, 2011).

Introduction to the Model
We adapted the DTLE concept to incorporate the principles of self-regulated learning (Nicol & Macfarlane-Dick, 2006) and the GOALS framework (Orsmond et al., 2011) to produce a practical web-based model for practitioners to improve dialogic feedback on student learning from assessments (Figure 1).
Figure 1.A model of dialogic feedback on student learning through assessment within a digital ecosystem architecture See section 3.2 for explanation of the component processes within the model.

Two Domains
The model consists of two domains in which tutor and student interactions occur.The traditional teaching and learning domain (1, in Figure 1) consists of the normal tutor-peer interactions that occur during lectures, seminars, tutorials etc. and students undertaking self-directed learning in the library or home.The second domain is the digital ecosystem (2) offered by Web 2.0's interactive functionality.

Learning Processes within the Traditional Domain
Here, the assessment task (3) triggers a learning process (4) which involves: understanding the general purpose and the requirements of the assignment task, and determining strategies to deliver the task.These processes are supported by the student: grasping the objectives of the assignment; appreciating how the assessment aligns with learning outcomes; referring to exemplars; addressing marking criteria; gaining advice from tutor's comments in lectures and tutorials; drawing from previous experience and feedback; monitoring and assessing own progress preparing the task through self-generated feedback; and discussing the task with peers (dashed-circle in Figure 1).All these processes are derived from the principles of self-regulated learning and the GOALS framework (Nicol and Macfarlane-Dick, 2006;Orsmond et al., 2011).The process of self-regulated learning is generated internally as the student monitors their own progress towards the completion of the assessment task.Externally derived feedback from tutors and peers also makes a contribution.
When the assignment is submitted (5), the tutor assesses, marks and provides individual written or verbal feedback (6) to the student.Feedback needs to be clearly understandable, constructive and encouraging whilst indicating the standard of work achieved and providing advice of how to improve future work.
All these processes occur in the traditional learning domain of the conceptual model (Figure 1).

Benefits Offered by the Digital Ecosystem Domain
The learning processes outlined in 3.2.2 for the traditional learning domain can be enhanced by the multimedia and interactive capabilities of Web 2.0. Figure 1 inlays a conceptual web-based digital ecosystem (2) accessed through a login (7).Popular social media and video websites are built around a user-friendly design architecture with easy navigability and high accessibility (8) and the digital ecosystem needs to emulate these features to provide a learner-friendly interface.To support the learning process through Web 2.0 social media, the digital ecosystem needs to offer (9): multimedia content in the form of audio and video materials; text-based documents such as web-pages, Word files and PDFs; communication tools for comments, rating and messaging; uploading learner-as well as tutor-generated materials; the option to assume an alternative identity through a pseudonym; and functionality for students and tutors to interact actively or passively, learning from other people's comments.This two-way communication not only allows tutors to feedback to students but also allows students the opportunity to respond to the feedback (10).In this way, tutors can see if their comments have been properly understood thus providing them with the opportunity to improve the quality of their feedback if required.
Provided that content is not removed, the legacy of materials in the digital ecosystem ensures that learners can view and learn from previous cohorts of students.

Development of a Web-Based Correlate of a Digital Ecosystem to Support Learning from Feedback
As the Web 2.0 functionalities in our digital ecosystems model are standard features within sharing-based websites such as YouTube, Facebook, Twitter, and Wikipedia, we wanted to explore bringing together the salient features of Web 2.0 to build a discrete, web-based, password-controlled, low-cost, digital ecosystem as a correlate of our conceptual model with the following design specifications:  Support for digital multimedia (audio and video) to enrich the user experience;  A web-interface resembling popular video websites to ease site navigation;  Unique user accounts to personalise interaction within the website;  Asynchronous interaction to allow users to choose when to engage;  A discussion forum for communication between users;  Facility for users to upload and share learning resources;  Ability for users to feedback by rating and commenting upon content;  Password-controlled access to monitor students and tutors engagement.
Based on these specifications, a web-based system called ASSET was produced.ASSET was scripted in Perl, maintained on a server running the Ubuntu JeOS operating system with a backend MySQL database.The choice of programming language, platform and hardware were determined the authors' previous expertise developing Web 2.0 applications.

Controlled Access through User Authentication
Access to ASSET was through a login page where users entered their username and password.Users were appointed a role, principally student or tutor.Each role was assigned a set of permissions, determining the types of interaction associated with that role.User roles were defined by the site administrator.Because ASSET was LDAP-compliant, it authenticated logins through the university's database allowing users to use their university usernames and passwords.
ASSET's login page also contained an embedded video file welcoming users to the resource, explaining the site's purpose and informing users how to gain access.Thumbnails of recently watched videos were also provided.

Channel Webpage
Once logged in, users were taken to a channel webpage.Resources within ASSET were arranged into channels with each containing a set of videos supporting a particular module assignment.The primary resources within ASSET were short, 2-3min videos produced by webcam, saved to disk and uploaded by tutors to support student learning from assessment.An example screenshot is shown in Figure 2 below.Playlist videos could be viewed individually (A) on its own web-page, which provided information, such as a detailed description, tags, and popularity in terms of number of views (C).The webpage also allowed user-interaction (8, in Figure 1) including: adding the video to their personal playlist (B); rating the video on a five-point scale for quality and usefulness (D); sharing comments and reading other people's comments (E); and finding related videos (F).The rating and comments facility provide valuable feedback to the video's originator.
Because communication was asynchronous, tutors and students do not need to be logged in simultaneously, making engagement with ASSET more flexible.

The Digital Ecosystem Concept Applied to Teaching and Learning
The digital ecosystem model offers a way of conceptualising the complex interactions that occur between groups of people in the online space.As online and digital technologies progressively play a larger part in teaching and learning practice, a digital ecosystems approach provides a model of the learning process especially where it involves group learning and a complex interplay of relationships: between learner and learning materials, learner and tutor, and learner and peers (Laurillard, 1999).The inter-relationships between the 'animate' (i.e., students and tutors) and 'inanimate' (i.e., learning material and their physical media) share characteristics with biological ecosystems (Reyna, 2011).We found that the dialogic processes required of self-regulated learning from assessment feedback could be modelled within an ecosystem approach and, furthermore, be easily operationalized through a bespoke web-based software system called ASSET.

Pedagogy of the Assessment Feedback Digital Ecosystem
The pedagogy underpinning the model shown in Figure 1 is that learning from feedback is conceptualised not simply as acquisition of knowledge from tutors' feedback comments but more a process whereby students actively construct their own knowledge and develop their skills of learning from feedback through a shared process.This is in line with Shepard (2000) who viewed learning in the 21 st century as becoming more progressively constructivist but with assessment practice lagging behind and remaining focused on testing.Similarly, Leathwood (2005) saw that the process of assessment needing to be more socially constructed.By students interacting with feedback from assessments, they transform their understanding of the subject matter by discussing it with others, internalising meaning and making connections with what they already perceive to know.The tutor's role is therefore more to facilitate learning in an on-going dialogue rather than providing feedback long after the assignment has been submitted.Within a digital ecosystems approach, tutors also benefit from seeing reactions and responses from students.In this way, tutors assess the effectiveness of their feedback comments and hone their skills at giving feedback.An on-going and shared dialogue addresses the misalignment that sometimes occurs between students' perceptions and usage of feedback and tutors' intentions for the feedback, as highlighted by Orsmond & Merry (2011).

Contribution of Self-Regulated Learning
Models of self-regulated learning (Nicol & Macfarlane-Dick, 2006;Orsmond et al., 2011) largely describe the learning process of individual students within the traditional learning domain (Figure 1).In self-regulated learning the emphasis is as much on the process as on the product and involves individual students internally reflecting, monitoring and assessing progress, and obtaining feedback at each stage during preparation of the assignment.However, tutor support of individualised student self-regulation through traditional face-to-face approaches is prohibitively time consuming in terms of workload.The Web 2.0 features of digital ecosystems facilitate communication and sharing among communities of learners and tutors.This is important as students naturally collaborate and discuss their assignment tasks with close peers undertaking the same task.Such peer-to-peer learning can be highly beneficial but the number of peers that any one student can interact with is relatively few when using traditional face-to-face methods.A digital ecosystems approach using a social website up-scales individual self-regulation to group self-regulated learning.Our digital ecosystems model (Figure 1) incorporates a social networking approach allowing wider peer-peer and peer-tutor virtual interactions across the whole cohort of students or between one cohort and another.Involving a more diverse user population has several benefits.For instance as self-regulated learning requires drawing upon previous experience, for someone new to their discipline, there may be little or no previous personal experience to draw upon.Such learners would benefit from being part of a wider, more diverse learning network where those with previous experience are able to share their experiences with others from the same student cohort.

Understanding through Discourse
Assessment criteria, tutor comments and exemplars are usually provided to support students achieve their learning goals, but there is evidence showing considerable mismatches between tutors' and students' interpretations of assessment criteria and standards (Sadler, 2005;Weaver, 2006).These mismatches are less likely to come to light in the traditional learning domain because of restricted opportunities for discussion with tutors and peers but in the digital ecosystem, students can discuss their perceptions more easily with tutors and peers to clarify their understanding.An extensive body of literature has built up around the need for dialogue in assessment feedback which has been reviewed by Blair & McGinty (2012) who defined 'feedback dialogues' as 'a collaborative discussion about feedback (between lecturer and student or student and student) which enables shared understandings and subsequently provides opportunities for further development based on the exchange'.Two compelling reasons for the need for dialogue are that students find written feedback difficult to understand and that many lecturers find it difficult to explain what they mean (Chanock, 2000).Blair & McGinty (2012) suggested a conversational approach to tutor feedback whereby the student can expand on their ideas, ask questions, seek clarification and defend or explain their position.Such a discourse is a learning opportunity in itself and a digital ecosystem, such as offered by ASSET, not only promotes dialogue but allows it to be shared with other students in an efficient way and allows the sharing of a wider range of opinions.The facility to assume a pseudonym in ASSET encourages people to focus more closely on ideas, take more risks and participate more than in face-to-face discussions (Selfe & Meyer, 1991).On the other hand, when social cues are filtered through a pseudonym, the resulting anonymity may result in more self-centred and unregulated behaviour though Dwyer (2007) concluded that the use of anonymity and pseudonyms was a strategy for protection against negative social interaction.An advantage to having a bespoke, educational, password controlled system is that participants may not know the identities of all the other members but may be reassured that the other contributors are members of the same student group and the focus is on learning rather than random issues that might inflame unregulated behaviour.

Operationalising the Digital Ecosystem through ASSET
In addition to modelling self-regulated learning from assessment within a digital ecosystems framework, we successfully operationalized the model to produce a web-based system called ASSET with video-sharing and social media capability.Though many of the features of ASSET can be found on popular social media websites, we wanted to bring this functionality together to produce a discrete system which was academic in nature rather than purely a social site.The system was password-controlled as we wanted to restrict access to only those students and tutors involved in the assessment tasks.Login-access also allowed us to track user engagement and for users to customise the appearance of the website and content to their own preference.
In a previous study, we populated ASSET with multimedia files, particularly videos supporting assessment feedback across numerous disciplines and involved nearly 30 tutors and 300 students (Crook et al., 2011).In that study, we reported on the benefits of video feedback through ASSET to enhance stakeholder engagement with the feedback process (Crook et al., 2011).Questionnaires indicated that both staff and students perceived video technology as being advantageous especially in providing generic feedback.Furthermore, that video addressed some of the major issues with feedback provision, namely: lack of student engagement with feedback, staff workload, timeliness and quality of feedback received by students (Gibbs and Simpson, 2004).We also showed that video positively changed how tutors thought about and developed feedback for their students; and for students, video enhanced their active engagement with the feedback they received Crook et al., (2011).
The benefits of the ASSET digital ecosystem to support learning from assignments went beyond the beneficial effects of video as a medium for feedback.Rather, the communication tools within ASSET supported group discourse about feedback.If ASSET were to be used to support online group discussion while preparing an assignment, student learning would benefit as students would gain a better understanding of the nature of the task and the expected standard for the work (Rust et al., 2003).In ASSET, discussion occurs at the level of individual videos allowing the message from the video to be teased out for added clarification.Studies on YouTube have shown that the facility for commenting on videos allows a 'video-thread' to develop that is initially triggered by the video but takes various directions as comments are made on other people's comments (Adami, 2009).Likewise, ASSET feedback videos allow discussion to be triggered not solely based on the video but around comments on the tutors' feedback as well as on comments by other learners.The principle of allowing students to comment on tutors' feedback was applied by Gomez & Osborne (2007) who designed an assessment whereby students were assessed not just on the quality of their essay-writing skills but also on the quality of their written responses to the assessors' feedback.In this way, tutors saw if students understood the feedback given to them and how the students planned to act on the feedback for future assignments (i.e., feed-forward).Gomez & Osborne (2007) found that allowing students to formally respond to assessment feedback was of immense value in honing student skills in analysing and interpreting feedback.By seeing how students responded to feedback, tutors were able to improve the quality of their comments.
The episodic, varied and discrete nature of many assignment tasks, together with a long wait for feedback, are reasons why feedback on one assignment is often not applied to subsequent work.ASSET's repository feature allows generic videos and conversations about assignments to be saved from one year to the next, thereby serving as legacy resources and concurs with the finding that "the availability of feedback stored online for future reference augmented by the opportunity for, and expectation of, further dialogue provides the greatest benefit to future learning" (Hepplestone et al., 2009).
The way that videos were arranged and displayed on ASSET was based on popular video-sharing websites.
Video sites, such as YouTube, are highly user-friendly in terms of: ease of accessing videos, how videos are displayed on the webpage, and ability to search, bookmark, comment upon and find related videos.Feedback to the producer of the video in terms of number of views, rating and comments provide valuable information to producers and visitors.YouTube would not be as popular as it is, if the website showed its video content as a list of download links that the user had to scroll through, then download or play through a separate media player.
The milieu of a video-sharing site together with the sense of community interaction contribute to a successful digital ecosystem.ASSET's layout and functionality were designed to replicate the look and feel of social multimedia sites for these reasons.
Our evaluation of the ASSET video system in providing and supporting assessment feedback showed a high level of user satisfaction, with 80% of students reporting that they liked this system for obtaining feedback (Crook et al., 2011).This is in contrast with studies of more traditional forms of providing assessment feedback where students were either dissatisfied or ignored feedback in the learning process (Fletcher et al., 2012).There are many reasons for the popularity of ASSET, especially around the benefits of video which some felt came closer to a 'face-to-face' interaction with the tutor than written comments.The benefits of video for feedback are discussed at some length in our previous paper (Crook et al., 2011).Here, we have focused on the social networking features of ASSET that form a major feature of a digital ecosystem.Reports show that 73% of US teens and 47% of US adults use social networking sites (Lenhart et al., 2010) and online social networking is forming an increasing part of higher education.Studies have shown that the use of social media in university courses improves learning outcomes as well as helping students gain social acceptance and adapt to university culture (Yu et al., 2010).ASSET, through its social media features, allows students to upload videos, rate and make comments and thereby provides a platform for students to put across their views.The ability for students to respond to tutors fits well with Boud's contention (Boud, 2000) that: 'The only way to tell if learning results from feedback is for students to make some kind of response to complete the feedback loop (Sadler, 1989).This is one of the most often forgotten aspects of formative assessment.Unless students are able to use the feedback to produce improved work, through for example, re-doing the same assignment, neither they nor those giving the feedback will know that it has been effective' (Boud, 2000, p158).A major feature of our digital ecosystem model and the ASSET system is that it provides a facility for students to respond to tutor feedback as well as tutors being able to see if students have comprehended the feedback correctly.The participatory and interactive nature of ASSET allows for more varied types of assessment to be developed that take advantage of its social media features and that helps nurture self-regulated learning and the establishment of online learning communities (Dominguez-Flores & Wang, 2011).
At the time of conducting this project, there were few off-the-shelf web-solutions to delivering the functionality offered by ASSET, hence the need to develop a bespoke system.As Web 2.0 becomes the de facto state of the web, access to websites with multimedia and interactive capabilities are becoming easier and therefore the potential for applying digital ecosystems approaches within the curriculum to a wider range of learning activities may eventually become the de facto state of higher online education.

Figure 2 .
Figure 2. A screenshot of a typical channel webpage In ASSET

4. 4
Figure 3. Screenshot of video homepage of ASSET