A Study on Different Translation Evaluation Strategies to Introduce an Eclectic Method

“Translation Evaluation” (TE) is a delicate process. It plays a considerable role in the process of Translation Education. TE is a tool by which translation education could get its pre determined aims. The importance of TE through the last decades has led to many studies and researches in this field of study. Various strategies using tools and models of TE, based on linguistics and interdisciplinary fields, have been presented. The stimulus of moving from one strategy to another is to objectify TE more than before so that its findings become more concrete and supportable. But such an objectivism is more challenging. This paper has paid more attention to those challenging facets and showed to what extent this objectivism has been attained. Besides, some important strategies of the past and present based on five criteria of acceptable evaluation to signalize their shortcomings in the process of TE have been analyzed. A new procedural eclectic model of TE heeding the cited criteria has been introduced at the end.


Introduction
Evaluation of translation aims at analyzing and marking the translation drafts of students based on a specific theory translation.What is absolute in this process is the comparison of the Target Text (TT) with Source Text (ST), while what aspects of text (Linguistic, Paralinguistic or both) to be noted, and what tools and models of evaluation to be applied is an over changing fact of TE.Evaluation is noticeable form two points of view: (1) it is an effecting and significant part of translation education and, (2) it is a valuable tool by which the educational aims could be achieved.
By means of evaluation we could determine, on one hand, the suitability and usability of texts to be translated by students, and on the other hand, the translational and linguistic competence, and intercultural awareness of students.Therefore, appropriate education strategies could be driven from the TE`s findings.So, in plain words, evaluation is a stimulus to push forward the class to gain the delightful and pre determined level of proficiency.
Nowadays a variety of TE strategies is being used at the process of translation education, ranging from "traditional approach" comparing TT with its ST very subjectively and limitedly to "modern approaches" using new tools (e.g.parallel texts, corpora, testing frames) and models (e.g.functionalism, text typology, etc).In this article the most outstanding strategies of TE are being studied with regard to the five criteria of systematicity, comprehensiveness, validity, reliability, and objectivity.Then a new eclectic strategy is presented to optimize these criteria in its process.McAlester (2000) introduces four criteria for TE as follows: reliability, validity, subjectivity, and practicality (McAlester, 2000: 230-231, also see Garant 2009: 6).This study proposes that besides these four criteria, comprehensiveness and systematicity are essentially needed for TE so that it could cover all of the factors involved in translation systematically.On the other hand, practicality could be replaced with systematicity since one of the merits of systematicity is to conform the frame of TE to the situation in which the evaluation should take place in order to actualize evaluation.After all, practicality itself is a precondition of the five other criteria.That is, we cannot talk of the existence of these criteria in a strategy, unless they have been realized in the real world.
In order to realize these criteria in TE we should set the base of the introduced model on the fundamental point that different strategies should be applied in different stages of translation education process.When applying the strategies we should consider linguistic and translation competence of the students, and the pre determined purpose of the translation education in order to justify (and adjust) the frame of the ET with regards to these factors.
Before discussing the strategies and providing the new method, we should distinguish between the subjective oriented vs. objective oriented TE, and define the other four criteria in brief.

Is TE (To Be) Conducted Subjectively or Objectively?
Quantitatively speaking, evaluation of translation is essentially important, since through it the teacher could determine and mark the level of translation competence of the students, and study and describe their progress statistically.Despite such a necessity, TE is somehow a qualitative act.This is due to the fact that translation is a multifaceted process.In other words, translation involves other problems than language bound ones (which themselves are relatively concrete and provable), including terms of context, readership, topics, etc which are more extended and not at the disposal of the translators (and the evaluators as well).As a result, translation examination has become more challenging.Thus, evaluators have to intuit and generalize about the quality of the "abstruse" aspects.
As it could be deducted, qualitative oriented vs. quantitative oriented TE is a matter of subjective vs. objective conflict, respectively.Despite many theoretical studies (e.g.NewMrak 1988, Wilss 1996, House 1997, and Reiss 1977/1989) on TE, true objective evaluation is not possible.The main reason could be the lack of direct observation and description of personal, social, and discoursal factors of translation.The relative solution to this unattainability is to use a comprehensive and systematic approach to cover the "representations" of these factors (as much as possible), and to manage and take into account all of them properly in order to evaluate them in a valid and reliable way.
We believe that TE, besides being subjective, is more emphasizing to be comprehensive, systematic, valid, and reliable as much as possible.Comprehensiveness means TE should: (1) cover all the linguistic and paralinguistic factors of translation, (2) apply all the useful accessible tools in its process, (3) base its performance on appropriate evaluation models.
The three factors of validity, reliability, and objectivity are somehow meaning proof.Validity is the suitability of the method to the defined situation of evaluation to be verified.Reliability is the usability of the method to the similar situations to be confirmed.And objectivity means the validity and reliability of the method for the intended situation to be exemplified and proved.Thus the first two factors are prerequisite for the existence of the third one.
The last criterion of TE is the systematicity which is essentially needed in the new method.It provides TE with a defined framework, and releases it from its common chaos.Moreover, it is pre requested for stabilizing the four criteria mentioned above.In plain words, designing and demonstrating of the eclectic method is possible via systematic framework.When presenting the new method, we will pay more attention to this regulating aspect of TE.
Of course it should be noted that these criteria generally are not in an absolute but relative mood.It is a matter of different degrees of possibility.Now in order to find out to what extent each of the old and modern strategies of TE posses these criteria, each of them will be considered one by one.

Assessment of Existing TE Strategies
Regarding the aim, tool, and model of evaluation, this study categorizes the evaluation strategies into two types: traditional and modern strategies.Traditional strategy, comparing TT with ST, is focusing on structural, lexical, and orthographic aspects.While the modern strategies, besides the linguistic aspects, is more concerned with paralinguistic ones including terms of pragmatics, discourse, socio cultural context, power relations, etc.And what these strategies use as tools are variables ranging from dictionaries, different bilingual and comparable (or parallel) texts and corpora, to different kinds of testing.Translation model of traditional evaluation is preliminary of structural linguistics and of textual equivalence.While those of modern one include a variety ranging from functionalist, text typology, and skopos, to translational action.TE in the past was very limited, unlike the present which due to the modernity and technology developments the accessibility to experts, professional translators, computers, and new flexible testing formats are really easier than before.In spite of all these developments, the new strategies, like traditional one, have major problems regarding the criteria mentioned above.By considering all the strategies this will become clearer.

Traditional Strategy
This approach is the simplest kind of TE.It is a subjective comparison of TT with ST.It was usually done by translators, philosophers, philologists, and writers.This approach was common in the past by teachers in translation classes.The teacher evaluated the students' translation drafts by his limited linguistic intuition, knowledge and experience.What he looked for were concepts as textual equivalence, faithfulness, fluency, readability, neutrality, etc. Indeed each of these terms is debatable and needing special researches.Just relying on his knowledge, the teacher could not realize and analyze the terms objectively and comprehensively.Therefore, what is going to be yielded from his evaluation is not anything but the impaired translation evaluation and, as a result, "evolution" (For further reading see Pym 2009).
This strategy has been in use up to 1970, when pragmatic movements in linguistics and functionalist approaches (1980) and cultural turns (1990) specially in translation studies took place and the focus shifted from linguistic to paralinguistic aspects of the text.So, the text-oriented approach of translation education is replaced by processoriented.Now what is important and at centre of the class is not the teacher but the students and their interactions with the text to be translated.So, because of the increasing aspects to be analyzed, the process of evaluation becomes more complicated than before.As a result, new tools and models are needed for performing TE.

Modern Strategies
In the modern strategies, like traditional one, the TT should be compared with the ST.The base is the ST to which the TT should be conformed linguistically and paralinguistically.But the tools and models used in them are facing both: (1) semantics, syntax, lexis, and (2) pragmatics, functional equivalence, text typology, cultural norms, the skopos of TT in the target culture, etc.

Using Real World Sources
These sources include dictionaries, parallel texts, and consulting with experts and professional translators.Dictionaries and their usability in the process of TE as well as in translation (education) is the subject of many writings and researches.But these recourses of any kind (abridged/unabridged, bilingual/monolingual), in spite of some usefulness, have some limitations as evaluation aids: (1) They are voluminous and not portable easily, ( 2) their examples are decontextualized, (3) finding the needed information in them is hard and time consuming, (4) mostly they are out of date.
A good alternative resource is parallel texts.These texts are documents that have been produced independently in different languages by the experts of a subject field, having the same communicative function and containing professional terminology; linguistic patterns and concepts of that field of study.These texts remove many problems brought by printed dictionaries into TE.For example, the retrieval of information is easier, their samples are more context-based, and they cover more areas of the fields.
These texts, on one hand, are presented in translation class to improve the students` translation competence and, on the other hand, are helpful samples for teacher to evaluate students` translation.Totally, these texts show how a certain text type is produced in two structurally different languages, while aiming at the same function.Therefore, both students and teacher could get a comprehensive insight toward how a text type should be translated from one language to another while the ST function remains in the TT.
Consulting with subject field experts and professional translators is another good way to evaluation.Experts are familiar with terminology and concepts and language patterns of texts in their fields.Professional translators are familiar with the translation strategies to be applied to these texts.
Based on Bowker (2000) both parallel texts and consulting experts and professional translators have some drawbacks.Regarding parallel texts: (1) since the evaluator collects data manually from these texts, he could not gather a wide range of documents to ensure that all the relevant conceptual terms as well as linguistic patterns are present.(2) Due to its limitation, the human mind collects all the needed data, yet could not attain a conclusive idea upon the conceptual terms, and linguistic patterns (Bowker, 2000: 188-189).These drawbacks, regarding the strategy of consulting experts and professional translators, are: (3) many teachers think to get help from others makes their competence on risk, so they are reluctant to do this.(4) Moreover, teachers have limited financial resources for hiring experts and translators to do so (ibid).Therefore, this strategy although is more comprehensive than the traditional one, lacks enough validity and reliability.To enhance these criteria in TE, researchers introduced the second strategy.

Using Linguistic Corpora
Applying linguistic corpora in evaluating translation makes it more objective, comprehensive, valid, and reliable than before.This electronic tool enhances the accessibility to terminological and conceptual knowledge of different subject fields.In plain words, printed dictionaries, parallel texts, and the works of experts are turned to corpus format.Through these corpora, teacher can prove or reject the terminology selections by students and enhance the objectivity of evaluation.
The corpora exclude nearly all the shortcomings of previous strategies.Generating valid and reliable textual patterns are easy via the corpus.Since these corpora contain hundreds of specialists' works, they are more excellent and cheaper substitutes than consulting.
What is important is to note that corpus should not be seen as a replacement for critical judgment, but rather as an aid to help evaluators to make sound and objective judgment.Therefore, this strategy is corpus-based and not corpus-bound.In plain words, the corpora should be viewed and applied as the efficient part of TE.Indeed objection to this approach rises from here: There is one criticism to this strategy, that is, the valuable data of these corpora follow no systematic framework to be inserted in and sampled for TE.In our new strategy this problem would be removed, and systematicity will be emphasized considerably.

Using Testing Formats
The other approach which has been used is adopted from systematic testing of language education.Using this tool in translation classroom clarifies up to what extent the students have reached the educational aims.This kind of evaluation is part of decision making process, methodology, and syllabus designing.It makes a great effect on the students' learning experience, understanding of SL-TL differences, and enhancement of their translation competence.
Testing includes different kinds (placement, diagnostic, Progress, etc.) and types (yes/no, multiple choice, matching, etc.).Each of these has special usage throughout the translation education process.Validity and reliability are of major interest in testing.Systematic marking is another one.The level of objectivity is enhanced (but just in evaluating linguistic aspect).Like other approaches, this one has its own drawbacks as well.(1) Adopting language education system of testing to TE returns evaluation to sheer traditional linguistic-bound approach.The discrepancy is that the latter is more objective, systematic, valid and reliable than the former.(2) Testing, due to its cited limitation of scope, is not comprehensive.
By considering the traditional and modern strategies, which one do you think is superior?The traditional approach subjectively focuses merely on linguistic dimension.Parallel texts and dictionaries, despite their relative objectivity, are not valid and reliable enough.Although these two features are enhanced in the corpus evaluation, this kind of TE lacks systematic marking which is common in testing.Even though the last approach has contained the above-mentioned criteria, it is not comprehensive.That is, it just addresses the linguistic elements of translation, and ignores the paralinguistic factors.
So, the answer, we believe, is an eclectic strategy from those strategies mentioned above, so that the drawbacks of each one would be removed or amended.In accordance with the five evaluation criteria, this approach becomes a situation-based one.Translation trainees use many different tools and models of translating based on their aim, competence and situation of translation.Their evaluation of their translation should not be fixed and unchangeable, but flexible to the situation in which the translation has been done.Therefore, different tools and models could be used for TE.
Evaluators should bear in mind that each tool and model is suitable for a special situation.On the other hand, not all tools and models can be used with each other.For example, by relying on his personal intuition, an evaluator cannot determine how much and to what extent the function of ST is transferred to TT.But he could do it via corpus-based approach and text typology model.Therefore, the co-working of tools and models, and their suitability for pre determined purpose of TE is to be respected; otherwise the evaluation lacks enough validity.

A Prelude to Eclectic Method of TE
We claim that TE is a long-term process consisting of some essential stages.Each of these stages includes different combination of tools, models and aims of evaluation in a meaningful and continuous mood.These stages are designed according to syllabus and predetermined purpose of each stage of translation education process.
The dominant factor in the process of translation education is "Translation competence".Toury (1984) divides translation competence into three parts: (1) Bilingual competence which refers to mastering to both SL and TL in all linguistics levels, (2) Interlingual proficiency means to know how the two languages are similar and different from one another, (3) Intercultural transfer competence that is to be able to transfer the ST to TT in a sociocultural context (Toury 1984: 189-190, also see Malmakjar, 2008: 302-303).Translation competence evolves via the process of theorizing (borrowed from Robinson 1997) from one level to another.According to Robinson, the central idea of theorizing phenomena is the fact that the students being exposed to different situations of translation activate and reconstruct their own knowledge and understanding of translation phenomena (Robinson 1997: 124).Thus, the higher the level of translation competence of the students, the more linguistic and non linguistic aspects of translation will be exploited by them.
We borrowed Toury's three part division of translation competence as the base of three stages of translation education process.We believe that as the overall process of translation education at last leads to the final goal (that is competent translator), each individual stage follows its certain aim (mastering bilingual competence, interlingual proficiency, and intercultural transfer) parallel to that final one.In each stage different tools and models may be applied.
Through adopting the same policy to TE, we design a stage by stage evaluation process that accurately fits the frame and strategies taken for translation education since translation competence is the most related factor to TE as well.That is, we indeed evaluate the translation competence of the students.So, in order to evaluate the ever-changing aspects of competence we should address them one by one, and use the appropriate tool and model of evaluation at any stage.

Process of Eclectic Method of TE
This study divides translation education course into two major parts: (1) Microlinguistic and (2) Macrolinguistic.At the first part the language systems of SL and TL and their differences and similarities at (semantics, syntax, lexis, etc.) levels would be taught to students so that the bilingual and interlingual competences of them improve.Because of emphasizing textual features by students through the process of translation, theories to be applied for this part are generally structural ones like that of Jacobson`s1959, Catford's 1965, etc.
Observing the good control of students on the microlinguistic aspects of SL and TL, teacher embarks on the second part.Here a translation phenomenon is provided with a situational and functional approach and the focus shifts from the translation text to the process of translating.In this process the students become aware of the fact that translating is a socio cultural activity.Now they practice to enhance their intercultural transfer competence which aims at contextualization of translation into the receptor culture and, as a result, the application of functional theories (See House 1977/1997, Reiss 1977/1989, Reiss & Vermeer 1989) becomes essentially necessary to this stage.In this stage they learn how to produce functional equivalence.Subsequently this would be explained more.
Teachers usually use both traditional and modern strategies to evaluate the students` translation competence.But none of the strategies could satisfy the teacher or the students.What is essentially ignored while being vital is the conformity of evaluation strategy to the principles of each education stage.

Evaluation of Bilingual Competence
The first step of evaluation in translation classroom is done like that of language education.Testing is a good option for assessing the students' abilities on SL and TL separately.As mentioned before, testing is linguistically objective, valid, reliable, and a systematic strategy.Thus, it is suitable for assessing the novices translation.What is considered here is such topics as structural, lexical, and orthographic correctness.Theses will be evaluated well by different types of language tests.

Evaluation of Interlingual Competence
After relative mastering of structural, lexical, semantic, and syntactic levels of both languages, students should go ahead and perceive the differences and similarities between the source and target languages.The more different languages, the more challenging this stage would be.This stage, welcoming to compare and contrastive analysis, is the starting point at which the two languages involved in translation education encounter each other.
At the first phase of this stage integration of traditional approach into testing format is recommended.Since the selected texts to be translated by the students are structurally and lexically the simplest ones, by relying on his knowledge the teacher could identify mistakes committed by the students.Integration of the traditional approach is vital to provide the TE with systematic and concrete marking of student`s drafts.It should be noted that the purpose of this stage is nothing more than the sheer assessing of students' realization of simple differences of the two language structures and lexis.
At the more developed phase the integration of bilingual corpus-based approach with testing is suitable.Bilingual corpus is electronic collections of STs and TTs.The purpose of evaluation is to see how much the students are aware of textual equivalence at micro linguist level between ST and TT.Because the selected texts to be translated are more complicated and realistic than those used in previous stages, the teacher should be supported by resources like corpus texts emerged from the real world.Like previous steps, these texts should be designed in testing formats.

Evaluation of Intercultural Transfer Competence
Competences of the students at this stage guide them to contextualize the translation process.They see that translation follows a special skopos and function, has its particular readership, is affected by the power relations in the society, and is an ideological affair supported (or rejected) by social institutions.Thus, assessing the students' awareness and application of "contextualization" in translation should not be missed out.This would be done through placing the students in challenging contexts of translation, where they should identify the source text function and reproduce it in the target text.
Considering factors like pragmatics, subject field, text function, text type, and readership is necessary in the macro linguistic level of translation education.Since these factors are "invisible" for the evaluator, and not accessible to him.Therefore, they are more challenging than the concrete textual factors.To evaluate these factors there is no way except assessing their "representatives" in the translation draft.These representatives may be special terminologies, linguistic, esthetic, or stylistic patterns, etc.So, the teacher should recognize them and consider them into his evaluation, that is, to see to what extent the students are aware of these contextual factors, and how this knowledge is represented in their translations.But as mentioned before, the teacher could not do it by himself.So, he should resort to some sources.
The best source is parallel texts in the form of corpora.Due to its lack of systematicity, these corpora should be integrated with the testing format.This means that the corpora samples should be presented to students in the form of tests.Up to now testing has been used just for assessing the linguistic elements, but now its application will be extended to nonlinguistic ones as well.For example, if the purpose of evaluation is to assess the functional dimension of the translation, it is appropriate to use the text typology model and, therefore, extract some texts from the parallel corpus and tailor them in a testing mood.As said before, both students and teachers benefit from the corpora as a whole.They become familiar with the special textual patterns, terminologies and concepts that are used (differently in two languages) for a special function.As a last word, the two approaches of testing and corpora could work together and form an eclectic method of objective, comprehensive, systematic, valid, and reliable.

Conclusion
The past and present TE strategies are being used at the process of translation education, ranging from "traditional approach" comparing TT with its ST very subjectively to "modern approaches" using new tools and models.In this article, the strategies are studied based on five criteria of systematicity, comprehensiveness, validity, reliability, and objectivity.Assessments of the existing TE strategies are summarized in the following All of the contents illustrated in the table were explained in paper body in details.So, it is time to step into the last stage of the study, that is, summarizing the eclectic method of TE.This is illustrated briefly in the following table: Testing is a good option for assessing the students' abilities on SL and TL separately.

Evaluation of Interlingual competence
First phase of this stage: integration of traditional approach into testing format.
At the more advanced phase the integration of bilingual corpus-based approach with testing is suitable

Evaluation of Intercultural transfer competence
The corpora samples should be presented to students in the form of testing.
The main reason to introduce this new method was to overcome the shortcomings of the TE ongoing strategies with regard to the cited criteria.Thus, as shown in the table, we designed a stage by stage evaluation process to assess the students' translation competence.To evaluate the ever-changing aspects of competence (ranging from bilingual and interlingual to intercultural competence) we addressed them one by one, and proposed appropriate strategies or (combination of them) for any particular stage.That is why it is entitled eclectic method.

table : Table 1 .
Existing TE Strategies

Table 2 .
Process of Eclectic Method of TE