Analysis of the Process and Outcomes of Eportfolio Development to Recognise Formal Learning in Zimbabwe

The process of developing electronic teaching portfolios can be used to evaluate the teacher's competency and guide a long-term professional development. This paper addressed the issue of assessment that is linked to the demand for accountability and standards through use of ePortfolio system. The ePortfolio system is then used as an authentication measure for students' work. The study survey is based on two groups selected from a local university; one group used paper portfolio and the other used electronic portfolio. Data was then analyzed from these two perspectives and digital story telling. EPortfolio development involved defining goals and context of the case, the collection of artifacts, selecting relevant information, showing a reflection and a projection of how the results are produced. Results showed that ePortfolios can be easy to be designed and implemented as a learning tool.


Introduction
The purpose of this study was to analyze the effectiveness of electronic portfolios in the assessment of student's work in a formal learning environment relative to paper based portfolios.The researcher explored the process of preparing an electronic portfolio using computer and multimedia technology.The final ePortfolio system was analyzed for evidence of self reflection and self assessment.The strengths and weaknesses, as well as the software and hardware problems that students encountered during the ePortfolio process, were also examined.

Nature of Study
The researcher conducted the analysis by implementing a case study based on the comparison of paper based portfolios against ePortfolios.This approach will provide two concepts to measure against hence providing conclusive results.The research was based on qualitative methodology.The researcher analyzed the major differences between paper portfolios and ePortfolios.Hypothesis H 0 : ePortfolio is a more effective tool to recognize learning than paper portfolio H A : Paper Portfolio is more effective

Project Participants
The participants in this study included:  Groups of students, i.e. paper portfolio and ePortfolio based  lecturers  computer lab technician  researcher

Choice of Target Groups
The researcher decided to use a selection of twelve students from Third year Computer Science students and Third year Industrial Manufacturing students.Students from either program were distributed equally in either group of paper portfolio or ePortfolio to avoid bias, as researcher speculated that if computer science students were used they tend to favor the idea of electronic media.
Students were encouraged to keep their names anonymous on the questionnaires as a way for them to feel free to air out their views with the exception of open interviews.The researcher also encouraged them to open up and not to feel pressured when making their contributions.

Data Sampling
Since the research was qualitative in nature a lot of data was involved hence the researcher came up with a number of techniques where data would be collected throughout the study.Techniques chosen depended upon the specific areas to be analyzed i.e. those, which highlighted the differences between the paper portfolio and ePortfolio.Questions or observations were then based on these factors.

Data Analysis Procedures
Qualitative inductive analysis was used in this study to discover critical themes emerging out of the data.The data analysis software package, Weft DQA (open source software), was used to manage narrative documents and transcripts, allowing for storing, browsing, indexing, and coding of all the text.Weft QDA allowed the researcher to explore documents and search for patterns and themes that emerged from the text.Search tools within the software were used to link, explore, and ask questions in order to determine relationships and establish hierarchies within the data.
The researcher created project using the Weft QDA package and loaded text files of questionnaires and interview responses, observations conducted during the study for analysis.Categories were created within the documents, hence proving some kind of coding on the responses.Documents were then marked for similar or exact words or category that appeared within them.

Assumptions:
 Technology is a major differentiating factor between ePortfolio to paper portfolio  Responses from participants were assumed truthful hence accurate interpretations were produced from the survey

Results
Questionnaires were created and distributed to the two groups of students for surveying purposes.It was necessary to get an overall view on how students are affected by the factor of technology hence a computer literacy questionnaire was created.

Computer Literacy (Student Perception)
To determine computer literacy and level of interest in technology a questionnaire was given to students prior to their participation in the portfolio project.Questions were designed to determine participants' access to computer hardware and software, level of computer activity, attitude towards technology, skill and activity level with different types of software, and personal purposes for using technology.The two groups of students were then asked to complete a computer literacy questionnaire.Their responses were then analyzed using Weft QDA.The researcher created two project files for ePortfolios and the other for paper portfolios for easier analysis of the data.Responses were marked for similar categories (See Table 1 and Table 2).The analysis was further done basing on the following statistical method, Student t-Test Analysis.Codes Category were further divided to positive and negative ones i.e. the types of category which would overally represent a more literate student and a less literate one respectively.

Computer Literacy -Positive Categories
Assumptions:  Data points for the ePortfolio and paper portfolio groups are not paired hence we use an unpaired t-test.
 Data follows a normally distribution as indicated by graph (see Figure 1)  Sample size per each group does not differ but is equal According to Figure 1, the ePortfolio group has more number of students per each category.For instance four students from the ePortfolio group have a high background in computers with respect to two students from the paper portfolio group.Four ePortfolio students are using a high version operating system such as Windows Vista with respect to three students from the paper portfolio.More students from the ePortfolio group seems to love computers as four of them have high computer attitude towards them unlike only two students from the paper portfolio who seem to love them. 1) H 0 : There is no difference between the populations of student attitude from which samples have been drawn; H A : There is a difference;  = 0.05 2) Test statistic, t calc = 1.35 3) Test is two tailed, since we have no firm basis to assume that there is a difference between the groups.Hence, degrees of freedom, df = (n A + n B ) -2= 12, where n A/B represents the number of students in the two groups being compared 4) Two tailed probability = 0.301 Looking up value of t critical from table where df= 12 and  = 0.05; Test statistic, t crit = 2.428 Since t calc = 1.35 and t crit = 2.428, such that t calc is less than t crit therefore the null hypothesis is accepted.
There is no statistically significant difference (at the 95% confidence level) between the ePortfolio and paper portfolio groups.Results obtained per each positive category do not vary much for the two groups.This shows that almost all students are computer literate the difference is on the level of literacy.

Computer Literacy-Negative Categories
Assumptions:  Data points for the ePortfolio and paper portfolio groups are not paired hence we use an unpaired t-test.
 Data follows a normally distribution as indicated by graph ( See Figure 2) There is no statistically significant difference (at the 95% confidence level) between the ePortfolio and Paper portfolio groups.The result obtained per each negative category does not vary much for the two groups.Hence some students from either group prove to show some negative attitude towards computers making them less literate.

Frequent Use of Graphics and Multimedia Technology (Technology)
The researcher decided that the two main items that differentiate the development of ePortfolio from general systems was the implementation of graphics and multimedia.Hence a sample of questions was created to get a survey on how frequently the sample group uses these two technologies.Use of graphics was to determine how student implement basic concepts like color, constructing graphical images, how flexible they are with designing say navigation buttons and text design.
Computer Literacy: Negative The sample group was assumed to be casual developers basing on the fact that most of them had not developed any software systems in the past.They were asked to state the type of multimedia software they had used it in the past.This helped to get a view if the students are using the latest software packages or any other package.
(See Table 3, Figure 3) From the analysis most students in the group are familiar with basic applications such as Word Processing, using emails and accessing the web, with zero students never using them at all.At most, one student has never had the chance to use any graphics or multimedia software, never done any research using the internet or even using database.Four students (majority) have used graphics or multimedia software.

Use of Multimedia and Graphics
Six students from each group completed a questionnaire on their usage of multimedia and graphics.This was necessary to deduce the experience and knowledge used as vital tools in portfolio development.(See Table 4, Figure 4)

Sometimes
All students within the group showed that they have somehow used the internet and email and were familiar with it.
Figure 4. Paper portfolio use of graphics/multimedia All students within the group showed that they had somehow used the multimedia techniques though the frequency on the usage was different.

Technology
Technology was discovered to be a major difference between paper portfolios and ePortfolios development process.Student showed to be computer literate though the ePortfolio group proved to be more literate than the other as shown from the questionnaire analysis.Hence both groups seemed to appreciate the use of technology and its importance.
Students expressed an interest in the creative and aesthetic aspects of the electronic portfolio process.They appreciated the appearance of the portfolio with colors, text changes, background graphics, animation, and drawing.One student from the ePortfolio group decided to design his own portfolio using the same concepts.
Student Attitudes towards technology:

Positive Comments Negative Comments
It was easy, fun, simple, and I was successful at producing my own products.
I was really worried about it because I don't know how to use technology.
I had fun in using with Wiki spaces and HTML.It was frustrating to look at all the things you have to do.
Electronic interfaces are great.I consider all portfolios either paper or electronic a waste of time.
I can apply this knowledge to other areas.

Project really stressing
Technology makes our work easier A lot of time is spent understanding the concepts

Evaluation of Experience During Portfolio Development
Besides observation technique the researcher also formulated a number of questions, which were given to the students to determine on what they have experienced during the ePortfolio development process.These questions touched on how data is collected, how it's analyzed and how results are obtained from the portfolio development.
Questions were evaluated using the following marks 1-Disagree, 2-Undecided and 3-Agree.They were a total of 7 questions and the maximum obtainable mark was 21.
Responses from twelve student participants were taken down and analyzed.The highest mark of 21 showed the most gained experienced student whilst 7 was the lowest mark showing that during the ePortfolio development a student would have gained nothing at all.Each student's form was analyzed and a total mark was calculated.The marks are represented in a table (See Table 5, Figure 5).

Rarely 17%
Sometimes 17% Responses of the twelve student participants were taken down and analyzed.The highest mark of 21 showed the most gained experienced student whilst 7 was the lowest mark showing that during the ePortfolio development a student would have gained nothing at all.Each student form was analyzed and a total mark was calculated.The marks are represented in the Table 5. Unpaired t-test between means  Data points for the ePortfolio and paper portfolio groups are not paired hence we use an unpaired t-test.
 Data follows a normally distribution as indicated by graph( See Figure 5)  Sample size per each group does not differ but is equal Figure 5 shows that students from both groups i.e. ePortfolio and paper portfolio managed to gain experience in developing the ePortfolio system.
Therefore it is suitable to perform a t test to determine statistical significance i.e. determining if the result found from the sample will represent the entire population of thirty five students (Computer Science and Industrial Manufacturing).Hence being probable that a similar relation would be obtained if the experiment were replicated with other samples drawn from the same population.
Figure 5. ePortfolio group experience marks Hypothesis: 1) H 0 : There is no difference between the populations of measurements from which samples have been drawn; H A : There is a difference 2)  = 0.05 3) Test statistic, t calc = 0.728 4) Test is two tailed, since we have no firm basis to assume that there is a difference between the groups.Hence, degrees of freedom, df = (n A + n B ) -2= 10, where n A/B = the number of students in the two groups being compared 5) Two tailed probability = 0.4832; Looking up value of t critical from table where df= 10 and  = 0.05; Test statistic, t crit = 2.228 Since t calc = 0.728 and t crit = 2.228, such that t calc is less than t crit, therefore the null hypothesis is accepted.
There is no statistically significant difference (at the 95% confidence level) between the ePortfolio and paper portfolio groups.Such that the result found from the sample can not represent the entire population of thirty-five students (Computer Science and Industrial Manufacturing).Hence it is not probable that a similar relation would be obtained if the experiment were replicated with other samples drawn from the same population.
In addition since there is no difference in the samples taken from the different population it emphasizes that the results obtained are more or less the same such that we can conclude that students from both groups managed to gain some experience during the ePortfolio development period.

Conclusion
The ePortfolio designed acted as a guideline for the student participants to develop their own ePortfolios.This aided three students from the ePortfolio group to design their own ePortfolios, implementing what they learned during the ePortfolio development process.This showed that it is possible to implement the idea of ePortfolios in student learning The computer literacy questionnaire proved that all student participants from both groups were computer literate even though six of the students from the ePortfolio group seemed to be more literate as they were well versed with a lot of computer technology.
The issue of technology was something interesting to the students as they appreciated its existence and how it improves the portfolio from paper to electronic.Six students had a negative attitude towards the whole idea of technology.The researcher was able to air out its importance as it was implemented perfectly in designing the ePortfolio.
Students were able to reevaluate the researcher's ePortfolio and the points produced generally showed that the design of the portfolio was well done.The results were not statistically significant hence showing that a different result was likely to be obtained if a different sample was chosen from the population hence this was a limitation of the study.
A total of nine students also evaluated the portfolios produced by the other three students and the results produced a small standard deviation such that the probability of error was low hence proving validity.The marks awarded basically showed that the ePortfolios were done satisfactorily.In conclusion basing on the analysis of the results obtained it is feasible to implement the concept of ePortfolio as a tool to recognize learning such that we accept the null hypothesis.(ePortfolios are an effective tool to recognize learning)

Challenges in ePortfolio Design
 Developing learner centered applications It was a challenge to come up with an ePortfolio that supports learning processes as it was slightly easier to use already Open Source portfolios such as Wiki spaces.The researcher managed to come up with an ePortfolio designed to record and support problem centered learning. Supporting the recognition of learning and reflection on learning One main aim of the research was to deduce whether the ePortfolio is able to recognized formal learning.This brought a great challenge as most studies have shown ePortfolios as being weak in their support for both the recognition of learning and reflection on  Sharing learning content Another important aspect was to design the ePortfolio in such a way that it recognizes the importance of ownership and sharing, not just through flexible and powerful permission environments.

Recommendations
All students involved in this study made the recommendation that students in future classes be given sufficient time to use the technology in advance, before development process.Twelve of the students felt stressed because they were not comfortable with computers.The advice that students offered included the following:  Start as early as possible so that you can gain confidence in the technology  Play around on the computer  Explore some software

Table 1 .
Computer literacy categories Test is two tailed, since we have no firm basis to assume that there is a difference between the groups.Hence, degrees of freedom, df = (n A + n B ) -2= 12, where n A/B represents the number of students in the two groups being compared 4) Two tailed probability = 0.301; Looking up value of t critical from table where df= 12 and  = 0.05; Test statistic, t crit = 2.228 Since t calc = 1.21 and t crit = 2.428, such that t calc is less than t crit therefore the null hypothesis is accepted.
30 Sample size per each group does not differ but is equal Figure2.Computer literacy: negative According to Figure2, the numbers of students who do not have either a computer at home or full internet access i.e. a modem is the same from each group i.e. there are five students from each group.More paper portfolio students have a less background in computers, are using lower version operating systems, spending less time on a computer with five of them having a low attitude towards computers, i.e. they do not really love them.Therefore we can use the Student t-Test for analyzing statistical significance.0 : There is no difference between the populations of "student negative attitude" from which samples have been drawn; H A : There is a difference;  = 0.05 2) Test statistic, t calc = 1.21 3)

Table 4 .
Paper portfolio group: Use of technology