Performance Evaluation of Organizations : An Integrated Data Envelopment Analysis and Balanced Scorecard Approach

The method that we propose in this paper tries to improve the traditional data envelopment analysis (DEA) by proposing an artificial alternative as an “ideal unit” in order to evaluate and rank the other alternatives in proportion to this ideal unit. Our approach integrates the balanced scorecard (BSC) by DEA and develops an extended DEA model. The input and output measures for the integrated DEA–BSC model are grouped in “cards” which are associated with a “BSC for organizations”. The BSC is embedded in the DEA model through a hierarchical structure of constraints that reflect the BSC balance considerations. Using a quantitative research method, we illustrate the proposed approach with an example.


Introduction
Most of managers are seeking ways to increase productivity in their subsystem.Thus, productivity assessment in organizations seems to be very important.As a matter of fact, productivity includes to main factors: efficiency and effectiveness.Therefore in order to assess productivity, mangers have to control these two parameters.Too many methods had been developed to assess organization's performance but the complication of modern organizations and their dependence to knowledge and information make traditional ways unsuitable.In this situation first, BSC was introduced as a new performance evaluation method, then it sets forward as a new approach to achieve strategies.This method has been very effectively in many organizations so that most of successful companies and institutions were ranked in fortune magazine, use BSC method as a powerful achievement tool.
On the other hand one of the techniques which is a good one to measure efficiency is data envelopment Analysis which tries to maximize efficiency by taking consideration of inputs and outputs.It is a mathematical programming technique that calculates the relative efficiency of multiple decision making units (DMUs) on the basis of observed inputs and outputs, which may be expressed with different types of metrics.The basic concept in DEA is to measure the efficiency of a particular DMU against a projected point on an "efficiency frontier".The usefulness of DEA in evaluating multi-criteria systems and providing improvement targets for such systems is expressed in the large number of its reported applications.
In this paper unlike CCR method which measures relative efficiency of units, we present a model to measure absolute efficiency of each unit in proportion to the best existence alternative.Thus, all inputs and outputs of DEA model will be determined by BSC perspectives first and then by performing DEA model we can assure that organization will move through its vision and achieve maximum efficiency as well.This new method can be useful for mangers and decision makers who are involving in process of organizational performance assessment and can be lead to improvement for those who applied lessons from the evaluation.
The rest of the paper is organized as follows: Section 2 provides a literature review.Sections 3and4 describe BSC and DEA approaches respectively.The integrated DEA-BSC model is presented in Section 5; Section 6 discusses an example that applies the DEA-BSC model.Finally, Section 7 presents concluding.

Literature review
Over the last few decades, the problem of performance evaluation has attracted significant attention that has led to a variety of methods.These methods seek to develop quantitative measures to assess the performance of organizations by systematically obtaining and integrating subjective and objective data.The methods range from simple screening procedures to sophisticated mathematical procedures.
More recently, some researchers have proposed DEA as a tool for evaluating organizations such as banks for instance Bergendahl & Lindblom (2008) used DEA as method for evaluating the performance of Swedish banks according to service efficiency.And Giokas (2008) proposed a new model for assessing the efficiency in operations of a large Greek bank.
The other researchers provided DEA model for the efficiency and effectiveness in railway performance (Ming & Erwin, 2008).They categorized the relevant measures of the evaluation as either inputs or outputs of the DEA model and used the efficiency scores to rank order the organizations.And finally Chen & Chen (2007) proposed DEA performance evaluation for semiconductor industry and also Eliat et al. (2008) suggested an integrated DEA-BSC approach for R&D projects evaluation.The method that we propose in this paper tries to improve the traditional DEA model (CCR) by proposing an artificial alternative as an "ideal unit" in order to evaluate and rank the other alternatives in proportion to this ideal unit.In this regard we propose the literature review of BSC and DEA methods.

BSC template
The balanced scorecard is a strategic planning and management system that is used extensively in business and industry, government, and nonprofit organizations worldwide to align business activities to the vision and strategy of the organization, improve internal and external communications, and monitor organization performance against strategic goals.It was originated by Drs.Robert Kaplan (Harvard Business School) and David Norton as a performance measurement framework that added strategic non-financial performance measures to traditional financial metrics to give managers and executives a more 'balanced' view of organizational performance.The balanced scorecard has evolved from its early use as a simple performance measurement framework to a full strategic planning and management system.This new approach to strategic management was first detailed in a series of articles and books by Drs.Kaplan and Norton.Recognizing some of the weaknesses and vagueness of previous management approaches, the balanced scorecard approach provides a clear prescription as to what companies should measure in order to 'balance' the financial perspective.The balanced scorecard is a management system (not only a measurement system) that enables organizations to clarify their vision and strategy and translate them into action.It provides feedback around both the internal business processes and external outcomes in order to continuously improve strategic performance and results.Kaplan & Norton (1996) describe the innovation of the balanced scorecard as follows: "The BSC retains traditional financial measures.But financial measures tell the story of past events, an adequate story for industrial age companies for which investments in long term capabilities and customer relationships were not critical for success.These financial measures are inadequate, however, for guiding and evaluating the journey that information age companies must make to create future value through investment in customers, suppliers, employees, processes, technology, and innovation." The balanced scorecard suggests that we view the organization from four perspectives, and to develop metrics, collect data and analyze it relative to each of these perspectives.The interrelationship between these perspectives is depicted in figure (1).

DEA template 4.1 History
Building on the ideas of Farrell (1957), the seminal work "Measuring the efficiency of decision making units" by Charnes, Cooper & Rhodes (1978) applies linear programming to estimate an empirical production technology frontier for the first time.Since then, there have been a large number of books and journal articles written on DEA or applying DEA on various sets of problems.Other than comparing efficiency across DMUs within an organization, DEA has also been used to compare efficiency across firms.There are several types of DEA with the most basic being CCR based on Charnes et al. (1994) methodology.However there are also DEA which address varying returns to scale, either CRS (constant returns to scale) or VRS (variable).On the whole DEA has been so many used in field of performance measurement.

Techniques
Data Envelopment Analysis (DEA) is a Linear Programming methodology to measure the efficiency of multiple Decision Making Units (DMUs) when the production process presents a structure of multiple inputs and outputs.Some of the benefits of DEA are: a) No need to explicitly specify a mathematical form for the production function.

b)
Proven to be useful in uncovering relationships that remain hidden for other methodologies.
c) Capable of handling multiple inputs and outputs.

d)
Capable of being used with any input-output measurement. e) The sources of inefficiency can be analyzed and quantified for every evaluated unit.
In the DEA methodology, formally developed by Charnes et.al efficiency is defined as a weighted sum of outputs to a weighted sum of inputs, where the weights structure is calculated by means of mathematical programming and constant returns to scale (CRS) are assumed.
The CCR model defines the relative efficiency of a specific DMU as the ratio between the sum of its weighted outputs and the sum of its weighted inputs.The variables, ur >0 and vi >0, are defined in a way that allows the unit in question to present itself in the most favorable way.The ratio, which is to be maximized for unit, leads to objective function shown in A (1) formulae.
The optimization problem in (A.1) is unbounded.Normalization constraints (one for each project) force the ratios of weighted outputs to weighted inputs of every project to be less than or equal to 1, as shown in A(2).
The constrained optimization problem defined by (A.1) and (A.2), including the positivity constraints of the weights, construct the original CCR model in its ratio form.The ratio formulation was transformed into an equivalent linear programming formulation known as the input-oriented CCR model (Note1) shown in (A.3).The constant is a small positive number that functions as a lower bound for the multipliers.
In this formulation, no a priori values are assigned to the input-output weights.This means, for example, that an alternative, which is a superior producer of a marginally important output, might be diagnosed as efficient even if it performs poorly with respect to all other outputs.Furthermore, in evaluating different alternatives, the same factors may be assigned widely different weights for the same dimension.

An Integrated DEA-BSC model
Two main factors of productivity meaning efficiency and effectiveness should be improved together, so that it can raise productivity.If an organization has high efficiency but doesn't gain effectiveness then it does not move in through its strategy.Otherwise the effort of organization doesn't result in profitability and goal achievement.
On the other hand if the organization's effort is efficient but not to be effective, it moves slowly to achieve determined long term goals.Since the effectiveness and strategy achievement measured by BSC and efficiency of an organization measured by DEA technique, so both factors of productivity (effectiveness and efficiency) can be measured by combining two models simultaneously.As a result, organization's perspectives can be defined and promoted by BSC technique, and then with regard to these perspectives, which organization is seeking to achieve will be seen in horizon.
Then inputs/outputs of ideal organization will be defined as which the organization has the best factors as a result to succeed.If the ideal organization enters to competitive arena with the others using DEA technique, it will definitely get the best point in terms of efficiency (1of 1) because it has the most proper factors to succeed.But the other organizations compare themselves to the ideal organization in terms of efficiency.While their efficiency is less than 100 percent they will try to approach to the ideal organization's specification by process changes, procedure revision, and using the other approaches in order to raise output.(Come closer to the ideal unit).
The decision makers have been always concerned about the definition of ideal unit.On the other hand one of DEA disadvantages is calculating relative efficiency and comparing DMUs with each other.By definition of ideal DMU, we will be able to compare the efficiency of other units to that of ideal point absolutely.
One simple way to define an ideal unit is to choose minimum of inputs of one type and select maximum output of its type as ideal unit specifications.So the model will be formulated as (A4).
To provide a new model, ideal unit specification and relevant constraint should be added to model, so unlike CCR model, efficient units (with the highest point) will not in result in several but one.

Numerical Example
In order to illustrate and motivate the method, we have developed the following example.Assume that there are twenty identical factories and all of them produce three kinds of products A, B&C.We consider raw material and salary as inputs of these factories, and products of type A, B&C as outputs.The relevant data are presented in Table 1.
-Solving the problem by CCR model: The problem should be formulized and then be solved without consideration of ideal unit specifications; the results are shown in Table 2.
-To dominate over the mentioned problem by integrated DEA-BSC model: While the model is solved by consideration of ideal unit specifications and adding constraint of 0 3 2 65 67 80 to the linear programming, the results in Table 3 will be achieved.

Conclusion
In this paper, BSC and DEA methods which have been approved as powerful tools to analyze and evaluate the performance in management science have been used.Then by combination of two models and applying it to real situation, the decision makers will be aware from the deviation of real performance of organizations in comparison with the determined strategic plan.
The integrated DEA-BSC model addresses three common goals that firms are trying to accomplish: 1) Achieving strategic objectives (effectiveness goal).
2) Optimizing the usage of resources in generating desired outputs (efficiency goal); and 3) Obtaining balance.The model is applicable for evaluating organizations (e.g., venture capital funds), as well as in non -profit organizations, such as government agencies charged with selecting R&D projects.
The integrated model we represent in this paper has capabilities of both DEA and BSC methods.If the efficiency of total units doesn't reach 1, this subject can lead to double motivation for the other organizations to make efforts in order to decrease inputs and raise outputs to get closer to the ideal organization.Any organization as ideal one is outstanding from strategic point of view, so the other organizations try to follow it in order to achieve both effectiveness and efficiency.The integrated DEA-BSC method advantages are as below: a) Definition of ideal unit: This item states that the efficiency of an organization can seldom reach to the highest point because inputs and outputs of ideal unit have been obtained through the best inputs and outputs selection of real organizations (artificially) but these results appeared in ideal organization, have come into a real existence in different organization.Therefore this matter causes to motivate continuous effort in order to acquire more efficiency and progress.b) Next advantage of ideal unit shows real specifications (inputs/outputs): The specifications have occurred to the real one before and simultaneous combinations of them have been taken place in ideal organization (artificially).As a result this matter avoids presenting unreal figures.In addition, other organizations won't be hopeless to achieve highest level.
c) The improvement of ranking system: unlike CCR model, efficient units (with the highest point) will not in result in several but one.So other organizations can be ranked based on ideal organization (efficient unit).This advantage causes to rank organizations more accurately and fairly.d) Motivating the other organizations: Only those organizations can perform efficiently (with the efficiency point of 1) that apply creativity and innovation to their processes, so these organizations strive enough to accomplish their routine activities greatly and achieve continuous improvement.This advantage makes the organization's staff to be motivated and in addition, to emphasize on continuous improvement and benchmarking.

Notes
Note 1.The CCR model admits both input-oriented and output-oriented models.While the input-oriented model focuses on savings of inputs, the output-oriented model concentrates on enhancement of outputs.To achieve our vision how will we sustain in our ability to change and improve?

Table 1 .
Inputs/outputs of factories in a constant period of 1 month It means that first factory gives 3000 $ to its employees every month, pays 5000 $ for raw material and produce 40,55&30 sets of products type A,B&C respectively.