Some Notes about Academic Evaluation: New Challenges for Accounting Scholars in Italy

This paper aims to analyze the recent evolution in the Italian accounting academic environment, discussing the behavioral, motivational and performance effects of such evolutions. In particular, the paper focus on the reforms in the area of academic evaluation in Italy in order to explain how it is changing, and why the local context can play an important role on the academic environment.


Introduction
Universities have come under increasing pressure to become key drivers of economic development in the age of the knowledge economy. In the case of Italy, there has been concern in recent years about quality and funding of academic institutions, but hardly any reference has been made about the impact of university quality on students' access to the labor market, and therefore on the secondary potential benefits deriving from the definition of appropriate incentives for research performance (Ciriaci & Muscio, 2014).
As sources of funding for universities and academic research are under ever mounting pressures, there is a growing demand amid public authorities of various countries for researchers to demonstrate the impact of their work on real world (Aidea, 2014). Many governments have placed priority on excellence in higher education as part of their policy agendas. Processes for recruitment and career advancement in universities thus have a critical role. The efficiency of faculty selection processes can be evaluated by comparing the subsequent performance of competition winners against that of the losers and the pre-existing staff of equal academic rank (Abramo et al., 2014). This paper aims to analyze the recent evolution in the Italian accounting academic environment, discussing the behavioral, motivational and performance effects of such evolutions. In particular, the paper focus on the reforms in the area of academic evaluation in Italy in order to explain how it is changing, and why the local context can play an important role on the academic environment. Some academics behaviors are indeed evident, such as focusing on publishing in journals indexed in certain databases not in international accounting journals, attending conferences that produce large conference proceedings on paper with ISSN number, not necessarily relevant accounting conferences (Geraci & Degli Esposti, 2011).
In particular, in Italian Accounting field, the assessment phase has been lived in a difficult because the Italian tradition is based on monographs in Italian and qualitative methods (Maran & Leoni, 2019). The first were penalized for setting the evaluation, the latter have been disadvantaged by the prevailing reference journals domain North American, always oriented quantitative methods. Dissatisfaction with the methods of implementation of the evaluation was then compounded by the fact that its results have had repercussions on the distribution of financial resources (Pavan, 2014).
During the work, we will follow a qualitative method, examining the Italian legislation and placing it in a conceptual framework, obtained by analyzing the international literature. The analysis that we intend to conduct is more oriented towards a popular article, highlighting the absence of several Italian journals on the international scene with a limitation of the diffusion of national research and a classification with accreditation of national journals in international rankings (Maran & Leoni, 2019).

Literature Review
In the past years, items such as the effects of assessment and evaluation practices, the matters associated with journal rankings, the relationship between accounting and practices and the academic career addressed in prior literature.
Following the Marxist concept of alienation, James (2008) analyses the accounting academic labour market and workplace conditions. Its particular focus is on the pressures, contradictions and dilemmas experienced by junior accounting academics at Australian universities. It is argued that, due to work, personal and family pressures, many junior staff members may struggle to complete their PhD within a prescribed time frame. Jones and Roberts (2005) underline that most articles published in top US accounting journals come from institutions based in the USA or a small number of other English-speaking countries. They have also been shown that the research paradigm favoured by US journals is financial economics, with the result that articles on accounting history or social and behavioural accounting are very scarce. European journals exhibit a more diverse content. Nevertheless, as shown by some studies, British authors are the main contributors to these journals On the same way, Raffournier and Schatt (2010) try test if the published literature is perfectly representative of the diversity of European accounting. They compare the content of 18 major academic journals in accounting over five years (2000)(2001)(2002)(2003)(2004) with the set of papers presented at the EAA congress in 2003, 2004 and 2005. The results give some support to the assertion that the diversity of European accounting research is imperfectly reflected in academic journals. They also are consistent with the idea that non-English-speaking scholars are at a competitive disadvantage in the race for publication in recognised periodicals. Reiter and Williams (2002) develop criteria for "good" scientific conversation, which leads to progress (defined as innovation and relevance). The key to this process is critical evaluation of background assumptions. The structure of scientific conversation in accounting and economics, whose theories and practices accountants adopted, are examined. They conclude that structural barriers result in a lack of adequate transformative critique, which contributes to the lack of progress in the accounting research program.
As regards the national context, Lowe and Locke (2005) analyzed the British case analyzing the perception of the quality of magazines, Cruz and Sanz (2008) analyzed the Spanish case focusing on the evolution of research evaluation, Venter and DeVilliers (2013) have focused on the South Africa by analyzing the influence of the accounting profession on academia, while Albu and Albu (2012) analyzed the case of Romania.
As regard the Italian context, it is interesting the paper by Greco (2014). He compares and discusses the periodic scientific research assessment systems of Italy and of four comparable Countries (United Kingdom, Australia, France and Spain). The key features of the above mentioned systems are discussed, including: the research quality assessment criteria, the reference to international research quality standards, the impact and relevance of the research outside the academia, the consistency of the scientific researcher's outputs with the scientific discipline and with the hosting institution's strategy and activities. The paper concludes with a direct comparison with the Italian system and with some possible proposals for improving the current Italian scientific research assessment framework. Abramo et. al (2014) present an empirical analysis concerning the recruitment procedures for associate professors in the Italian university system. The results of a bibliometric analysis of the hard science areas reveal that new associate professors are on average more productive than the incumbents. However, a number of crucial concerns emerge, in particular concerning occurrence of non-winner candidates that are more productive than the winners over the subsequent triennium, and cases of winners that are completely unproductive. Beyond the implications for the Italian case, the analysis offers considerations for all decision-makers regarding the ex post evaluation of the efficiency of the recruitment process and the desirability of providing selection committees with bibliometric indicators in support of evaluation (i.e. informed peer review). Reale et. al (2007) aim of the paper is to control the reliability of peer review when evaluating academic research in the Three-Year Research Assessment Exercise developed in Italy. Our analysis covers four disciplinary sectors: chemistry, biology, humanities and economics. The results provide evidence that highlights strengths and weaknesses of peer review for judging the quality of the academic research in different fields of science, vis-a-vis bibliometric indicators. Moreover, some basic features of the evaluation process are discussed, to understand their usefulness for reinforcing the effectiveness of the peers' final outcome (see also Manzoli 2014).

Conceptual Framework
The theme of the research evaluation has long faced by scholars of accounting (Adel Khalik & Ajinkya, 1983;Hopper and Powell, 1985;Chua, 1986). But only in recent years, the debate on the point has become stronger and more systematic. So if on the one hand a number of scholars have questioned the future of the discipline ijbm.ccsenet.org International Journal of Business and Management Vol. 16, No. 11;(Ahrens et al. 2008, others have begun to highlight the critical issues related to this process. Among these Khalifa and Quattrone (2008) show that some accounting research has been very prone to import from other disciplinary spaces different views on the academic and scientific craft (and in some accounting research traditions this is seen as the only way to make progress in accounting research), so much so that the implications for accounting research of the work by Kuhn (1970), Feyerabend (1975).
According Khalifa and Quattrone (2008) the process of defining what counts has become more democratic and this is certainly to be welcomed, as we will argue later. However, calls for the reorganisation of national academic systems (Mazza et al., 2008, with regard to the Italian case) may be based on superficial and stereotypical knowledge of what happens in other educational systems and a mythical reference to a supposedly unitary model which can serve as template for the reform. In Italy this has often been the British university model; in the UK, the American university system is often referred to as template for modelling local reforms.
Parker and Guthrie (2013) identify six main challenges for acconting academia. The first is the globalisation and commercialisation of higher education: many universities have moved beyond their traditional national focus to become players in a globally competitive higher education market. For instance, in the Australian higher education sector, the result has been commercialised higher education with academics becoming commoditised in the process.
The second challenge is the lack of funding for accounting education and research: departments are characterised by very high student/staff ratios, very large class sizes, heavy teaching, preparation and marking loads, limited time and support for research, and a casualised career structure.
The third challenge is the threat of evolving technologies and business models exclude the traditional model of a university: educators must concentrate on how to harness the advantages of technology in a business education environment.
The fourth challenge is that research in accounting is uncoupled from the concerns of the accounting profession, practitioners and future practitioners: there is a perception that academic accounting research is only relevant to other accounting academics and does not address real-world issues.
The fifth challenge is that accounting must overcome its negative reputation as a major contributor to the current global financial crisis: profession has been viewed as a provider of offshore financial services and tax avoidance measures.
The sixth challenge is that superficial forms of research quality assessment: journal ratings are becoming an increasing focus of national governments, government research agencies, university leaders and individual academics around the world.
On this last item, authors underline that technologies have been developed by several large publishing corporations to profit from these activities by producing calculative practices that can be used by governments to organise, measure and record research for all disciplines. These measures range across assessment of a nation state, individual universities, faculty, school, department and individual academic's research productivity. The contemporary effects of these activities penetrate every corner of academia. They impact on university funding, research grants, academic appointments, tenure and promotions.
According to Parker and Guthrie (2013) the university performance metric based research culture creates a market for research and researchers who can be commercially traded. Increasingly government systems for evaluating universities rely on publication media ratings, research funding revenue statistics and postgraduate research student graduation statistics rather than a direct focus on quality and significance of knowledge produced.
Parker and Guthrie (2013) also underline a risks of this new system: research designs and write-ups aimed at producing a perceived standard template to enhance prospects of acceptance in a targeted top ranked journal. The reduction in diversity of research topics, methods and perspectives as scholars rush to replicate the "successful" formula for acceptance in top ranking journals; the emergence of elites, who self cite and review each other's works; emerging scholars becoming mere research production workers for older scholars (and so on).
Following this approach, we will focus now on the Italian case.

The Italian University Context
The Italian university system is based mainly on state universities (67 universities), which receive 92% of the total enrollment (1.7 million). At private universities (29 universities, 11 of which telematics) is registered with 8% of the students. Over 40% of the students are writing to 11 major universities in the country (with over 40,000 students), and almost 70% is registered in one of the 26 universities "historical" founded before 1945.
The Italian university system in the last 15 years has gone through two phases clearly distinct. The first is characterized by a rapid expansion of the educational, economic resources and human ones, while accompanied by imbalances and overall problems of governance; the second, which began with the economic crisis, which has led to a significant reduction in resources and a rationalization of the training offer.
The recent university reform introduced by Law 240/2010 (so called "Reform Gelmini") has rethought governance and introduced, much later than other European countries, mechanisms of evaluation and accreditation of courses and the university.
According to Anvur (2014) since 2009 the total funding of the Ministry of Education in the university system has been reduced by about 1 billion (-13% in nominal terms, -20% in real terms). The reduction of resources has been made sustainable by reducing staff, particularly teachers' ordinary whose number had grown rapidly in the past, and the blocking of salary progressions. The teacher / student relationship was reported today at a high level. Over the next five years will come to retirement 9,000 teachers, 17% of the total; will be necessary to ensure the parts in order to safeguard the performance of the teaching load and Government of the universities and the research potential of the country.
Between 1993 and 2012 the share of college graduates in the population of working age increased from 5.5% to 12.7%; among young people aged between 25 and 34 years has gone from 7.1 to 22.3%. Substantial increases, showing that higher education is no longer limited to a narrow range of people, but it has become accessible to broad sections of the population. As in other countries, in Italy there has been a transformation of higher education, reducing its elitist character and opening the system to one that has been defined as a mass university.
In public debate this transformation, common to almost all advanced countries, was accompanied by a belief that even now in Italy there is an excess of graduates. Yet international comparisons show that not only Italy appears still among the countries with the lowest proportion of people in possession of a tertiary, even among the youngest, but also as the departure from the average European values will not be reduced over time, given the widespread expansion of university education in almost all countries (Anvur 2014).

The Evaluation of Research
In July 2013, the ANVUR published the results of the Quality Assessment Research (VQR) refers to the period 2004-2010. The Report VQR presented detailed data relating to the quality of the publications evaluated, developed at the area of science and then aggregated to the departments.
By analyzing the scientific production in this citation databases ISI Web of Science for scientific areas (bibliometric), and SCOPUS, for the areas of social sciences and humanities (not bibliometric), the Italian share of world total stands at 4.4 % in the first and 1.9% in the second. The presence of publications in international journals, mostly in English, of the areas humanities and social sciences is systematically lower than that of scientific areas, in all countries that are not native English speakers. In these areas, with the partial exception of the areas of Economics and Statistics and Psychology, the publications are mostly in the national language and magazine articles are often not the most important type of publication, given the importance of monographs and other types of publication.
Regarding the economic area, the share of scientific production in the Italian top 10% is lower than the world average. In Europe, Italy is in line with Spain, but it has a share of excellent products below that of other major countries. The share of Italian publications classified in the top 1% of the worldwide distribution in terms of citations to five years is below the European average, and the world of the major industrial countries; the Italian share is even lower than that of all the other European countries with the exception of Spain (ANVUR 2014).

The National Qualification: "Abilitazione Scientifica Nazionale" (ASN)
Articles 16 and 18 of Law 240/2010 provide for teachers a new recruitment system substantially in two distinct phases. The first one regards the procedure "Abilitazione scientifica nazionale" (ASN, National Scientific Qualification), through which the candidate is judged by a single committee at national level for each sector competition, which verifies the possession of the requirements needed to perform the duties of professor. The second phase spread in each university, where all the candidates holding a qualification scientific nationwide have the opportunity to participate in a special selection procedure (cd. "Call"), activated by the universities with the notice public competition.
The ANVUR defined the method of calculation of the indicators and procedures for the classification of journals. ijbm.ccsenet.org International Journal of Business and Management Vol. 16, No. 11;2021 55 The medians of the indicators, to be used as reference for the evaluation of candidates and commissioners, were calculated from information contained in the sites managed by Cineca. Within the procedure ANVUR then: a) has prepared the lists of candidates commissioners foreigners, to the extent of at least 4 per sector competition; b) has calculated, for each competition area the medians of all indicators, refer to the teachers, to be used for the evaluation of candidates for national scientific qualification For the calculation of the medians for sectors not bibliometric it makes reference to the following indicators, all referred to the ten consecutive years preceding the date of this announcement: a) number of books b) number of magazine articles and book chapters on c) number of journal articles in Class A.

The Implications in Academic Contest
Before starting with the analysis of the implications that the two processes mentioned above have had on Italian Accounting, it must first make it clear that in Italy Accounting is part of the more general context of Economia Aziendale. The scientific community of Economia Aziendale consists of 1,877 teachers and represented by the Italian Academy of Economia Aziendale (AIDEA).
The area of Economia Aziendale has been characterized by a widespread presence of products different from scientific article in international journals. The most important of these is undoubtedly that of the monographs.
Since the Economia Aziendale is a non bibliometric area, the peer review process and the identification and selection of peer play, as is obvious, the greatest importance. It is equal to be identified not so much (or not only) based on the role of researchers and academics, but mostly based on the wealth of knowledge and skills they possess. And 'this heritage that qualifies them as equal, specifically able to evaluate the products of scientific research of other researchers and academics.
According to SIDREA (the Italian Society of teachers of Economia Aziendale), the accounting policies adopted do not take into account the peculiarities of the Economia Aziendale. Scientific products such as monographs, which can have a significant value for the advancement of the discipline and for the dissemination of research, given the evaluation criteria defined and their weighting, were considered poorly suited to achieve the two judgments valuation higher. For example, 600 monographs submitted only 22 were assessed as good and none were judged excellent. Similarly, of 457 works presented as a contribution in volume, only 19 were judged good, and no one has been rated excellent. Journal in Italian languages with significant impact at the national level, both on the scientific community that the community economic and social, were considered quite marginal in the assessments.
According to Sidrea (2014, b) the use of medians influence the behaviors of stakeholders for certification resulting in a continuous increase of the values, then a competition quantitative, to the expense of the quality of research. Therefore, it should be defined minimum quality standards in the selection of scientific products for the calculation of the medians. As regards the medians of the monographs, a selection criterion easily applicable is to consider only monographs with a single author, publisher published by major national or international. The criteria set out relates only to the scientific production and should be understood as the threshold for access to the evaluation to the candidates for certification Following the paradigm of Parker and Guthrie (2013) we want to highlight the six elements that seem to characterize the environment of the Accounting Italian in the light of the above matters.

1) International journals
The goal of the academic Italian became publish in international journals with high ranking. Only this step can enable it to aspire to make a career and very often to obtain research funds from departments.
Obviously this aspect is generating a bit of confusion because as it is well known journal band A are limited, while the demand for products is very high.
Therefore, a possible trend of the market could lead to higher costs in order to publish the journals of high ranking, with the possible risk that the same scientific value of the articles scholar who can afford publication costs higher prevail.

2) Conference with ISSN code
Even on the issue of participation in conferences, the situation is very delicate. If in fact the past there were the usual annual events (conference of Aidea or the Europen Accounting Association) to which teachers used to attend, now the aim is to participate in conferences that recognize the paper also presented an ISSN or ISBN. It became this the pass to send or not the paper. Very often they prefer to present at conferences "minor" issuing a code ISSN, rather than major conferences (e.g. the EAA Annual Congress) that do not release that code.
Also are multiplying cases of conferences with the possibility of electronic filing and simultaneous recognition of ISSN code. In this case also saves the cost of travel which in a period of financial restraint can not always be supported.

3) English new language
As to internationalize it must write in English, it seems that the Italian is not used anymore. Increasingly, both at national conferences on both the Italian journals are present or published contributions in English, despite the audience of listeners or readers is made up almost exclusively Italian. It does not understand this trend actually, because if it is true that it has to be good at writing to publish abroad, should be just as aware to continue to write in Italian to continue a tradition of study that otherwise risks disappearing.

4) From the Book to the Article
Once in Italy all scholars of Economia Aziendale began his career writing a monograph in Italian. Now having to write a book seems a thought almost disappeared, because they start with writing articles, for use as stated in international journals. In this way risks disappearing tradition to bring forward their ideas through writing books, an operation that may allow you to be analytical, to study issues, connecting more arguments. The article on the magazine must be inevitably synthetic, so the risk is that the ideas are less extensive and not argued in detail.

5) More research less teaching
The lecturer should not treat only research, but also devote to teaching and service to the university and the region. In this phase the teachers seem too stressed on the research and consequently there might be a risk that the other two dimensions are neglected. This aspect should be carefully evaluated since the middle of the university are the students, who must be given a high level of teaching to avoid that they remain dissatisfied with the degree course. At the same time care must be taken because very often the places to teacher are allocated according to the needs of teaching, although the assessment of candidates is made on the results of the research. It may be the case to include in the assessments also about teaching, or to check internally from year to year as the teacher performs educational activities in the classroom.

6) A few research items
Finally, the concentration on the international research is leading to thinning of the research topics. In order to post on the most important magazines must adapt to the main stream, otherwise it is difficult that your work is appreciated abroad. This process could lead to the disappearance of research topics that have historically affected the Italian scholars of accounting (for example, the accounting history), but that at the moment seem to be undervalued by the current dominant. The latter is strongly influenced by Anglo-Saxon approach based on quantitative methods, while historically the Italian research was based on qualitative methods.

Conclusions
In the last few years have been reduced some imbalances and has brought the university system on a path of economic sustainability despite the decline in the resources available, the amount of which is on the whole unsatisfactory by international standards. Given the triple function of the university, teaching, research and direct support to the economic and social system, would require a broad reflection on the optimal size or at least minimum requirements of the university system and the resources to invest, as part of a governance renewed calls that all actors to respect the principles of responsible autonomy.
According to Albu and Albu (2012) publish in non-accounting ISI journals immediately, or let their research accumulate and try to get published abroad in accounting international even non-ISI journals, for international visibility in their field of expertise. This is a trade-off which is felt and resented currently by many Romania-based accounting academics, including the authors of the present paper. There will always be the danger of remaining the perpetual 'submitter' to international accounting journals According to Pavan (2014), the need for internationalization is shared and strongly supported, however it can not be exaggerated on the most prestigious international journals and it must live with the national dimension.
International journals are limited in number, which does not permit a "space" physical enough; they are also prevalent in the North American dominance where quantitative methods assume an emphasis that in many cases seems to prevaricate on the relevance of the objects and on the merits of the content. An orientation to them of a community of more than 1,800 professors and researchers is not physically realizable, and it would lead to a total flattening of research agendas with what is considered to be of interest in such environments.
Economia Aziendale has a strong tradition of science-based monographs in Italian. It is not desirable that they are totally marginalized in favor of journal articles. Necessary for them to proceed on the path of improvement paths through rigorous and reliable evaluation and acceptance.