Design of Performance Improvement Factors ( PIFs ) for Sustainable Development Indicators ( SDIs ) Metrics for Oil Sands Projects with Application to Surface Mining Operations Based on Continual Performance Improvement ( CPI )

The sustainability assessment approach utilized in the development of the WA-PA-SU project sustainability rating system includes three distinctive areas of knowledge: sustainability, continuous performance improvement, and multi-criteria decision-making. Previously, the study of sustainability led to (1) concluding the need for the development of a rating system for industrial projects, with a particular application to oil sands and heavy oil projects; (2) defining the structure of the rating system; and (3) assisting in the pre-selection of sustainable development indicators (SDIs) for surface mining operations. Assessing the sustainability of projects at certain points in time required the application of a methodology selected by the interested groups and/or stakeholders; however, measuring the improvement of projects in sustainable development performance over time (i.e., continuous performance improvement) presents additional challenges. Certain industries (i.e., oil & gas), projects (i.e., oil sands or heavy oil), or specific operations (i.e., surface mining) require a rating system with a particular level of flexibility, offering the opportunity for developers to improve the performance of operations, and for stakeholders to understand the difficulties-and benefits-of implementing SDIs and perform up to levels of truly sustainable development. The present manuscript introduces the performance improvement factor (PIF), which can be determined using three different methodologies: relevance factor or subjective stakeholder valuation, comparative assessment methods, and links to metrics. Additionally, the continuous performance improvement (CPI) indicator measurement is suggested and discussed for a pre-selected set of SDIs for surface mining operations in oil sands projects. Finally, a brief preamble discusses the proposed integrated approach for sustainability assessment and the part it plays in continual performance improvement, offering a foreword to upcoming manuscripts that discuss the other complementary parts of the integrated approach.


Introduction
Organizations and individuals are surrounded by measures.Individuals are expected to increase or evolve in performance to advance and accomplish their personal goals, and organizations must do the same to increase profit and confront aggressive market competition; therefore, performance measurement (PM) offers an opportunity to compare against a pre-established baseline to evaluate improvement over time.PM improves management processes, develops internal and external channels of communications, and provides fact-based decision-making tools (US Department of Energy, 1996).A key indicator of survival in competitive markets is, to some degree, defined by an organization's ability to measure the most critical processes and improvement (Malik et al., 2010).In a timely manner, organizations not only adapt but also react to change through measurement systems or metrics; furthermore, metrics are seen as effective tools to measure present performance and monitor possible upcoming risks (Srivastava et al., 2001).
Organizations and projects use indicators not only to measure current performance, but also to identify performance gaps between the current and pre-established goals, to show progress towards closing performance gaps, and to identify areas where actions are required to improve performance and close the gaps (Weber & Thomas, 2005).Moreover, information gathered in performance measurement processes is used for evaluation and planning, while, similar to indicators, performance measurement systems provide ways of communicating performance expectations, identifying performance gaps, and supporting decision-making processes (Hyland et al., 2007).Tarr (1996) adds to the debate a distinction between external and internal purposes for performance measurement, in which internal purposes consist of controlling and redirecting individuals and departments, assisting the process with feedback to adjust performance or targets, and offering a tool to compare performance against strategic and continuous improvement goals.
When it comes to performance measurement in the area of sustainability, the triple bottom line (i.e., environment, social, economic) is often measured using indicators.In the past, organizational improvement has primarily been a financial concern, but in recent years, social responsibility and sustainability performance have eclipsed finances as the areas where organizations face significant pressure to improve.Regardless of the nature of the businesses, organizations are being forced to demonstrate their commitment to continuous improvement in all three pillars of sustainability to survive in the business environment.Therefore, all industries are turning their attention to finding ways to measure the efficiency and effectiveness of their processes' alignment with the organization's objectives and goals, which must be a reflection of the stakeholders' needs.Environmental performance not only makes good business sense, but also economic and social aspects are included in organizations' periodic reporting.Measuring, managing, and communicating performance brings to organizations the opportunity to make the case for public acceptance to operate and obtain the so-called "social license".Organizations that actively report their performance understand process improvement, cost efficiency, regulations, stakeholder needs, and the market (Defra, 2006).
Organizations are not only starting to understand the concept of sustainable development, but also are also seeking to integrate it in their policies, mission statements, commitments, and business strategies (World Business Council for Sustainable Development, 2002b).In the progress, the responsibility to stakeholders and their needs have an effective impact on the manner in which companies approach the reporting of their operations; therefore, companies need appropriate systems to measure, control, and report their behaviour with a sustainability-oriented managerial approach (Perrini & Tencati, 2006).
Macro-level objectives for sustainability do not explain the transition to an effective decision-making process at the project level.The Bruntland Commission and other committees and organizations developed a series of macro-level objectives for sustainability.Additionally, existing frameworks, sustainability initiatives, strategies, and processes focus on global interpretations of sustainability in national or strategic objectives; however, a proven challenge for different construction professions relies on the ability to understand, translate, and achieve strategic sustainability objectives at the micro-or project-specific level (Ugwu & Haupt, 2007).
Following the measurement of performance, the likelihood for success is controlled through the monitoring and process, though the period between measurements must be considered the connection between goals and daily actions (Romaniello et al., 2011).Sustainable rating systems as continuous improvement and monitoring performance systems must be dynamic and flexible, measure performance in real time, show clear trends and goals, and reflect the stakeholders' needs; therefore, based on these elements, the rating system has the intrinsic possibility in its design to trigger a system of continuous improvement.Certainly, the design of indicators and their translation into metrics or measures requires translating the views of stakeholders' needs into business goals and objectives.Even further, to achieve sustainability, the process is required to include the development of customer-driven (stakeholder-driven) performance measures.Additionally, organizations typically substantiate their internal mission statement with a series of indicators (KPIs) to track performance against stated goals and objectives (Dalziell & McManus, 2004).
A group of sustainable development indicators (SDIs) for the surface mining industry indicates a step towards improving processes and sustainable operations; however, businesses and organizations are used to measuring financial and environmental performance separately.Given that improvement in non-financial measures drives financial performance (Kaplan & Norton, 1996), organizations and stakeholders are expected to promptly link the need for measuring performance of all three pillars of sustainability.Furthermore, companies are not only pressured to publicly report on their environmental and social performance, but they are also affected by their sustainable development (and its triple bottom line: social, economic, and environmental pillars) performance, which defines business success for organizations in the market arena (World Business Council for Sustainable Development, 2002).

Continual Performance Improvement (CPI)
"Clients, investors and other stakeholders are demanding continuous improvement" (Robinson et al., 2005).Organizations aim for survival in continuously-changing environments.Characteristics such as innovation and competitiveness provide organizations with tools to maintain their status or lead in the business arena.While continuous improvement methods support the competiveness of an organization, the culture of continuous improvement is associated with innovation (Hyland et al., 2007;McAdam et al., 2000).The performance measurement of the organization reflects the effectiveness of the efforts by the entity under evaluation (Zairi, 1993).Flexibility, responsiveness, and adaptability are factors affecting the decision-making process of organizations as stakeholders and customers influence change, demanding quick adaptation to their needs.However, meeting the requirements of a dynamic decision-making environment and adapting to continuous changes of strategic objectives are difficult tasks the organizations face; therefore, performance measurement systems and associated criteria must be constantly evaluated and updated if required (Cai et al., 2009).
Performance measurement of products and services includes the measure of cost, quality, cycle time, quantity, efficiency, and productivity, among others; however, the objective is not to collect data but to predefine performance goals and standards (Malik et al., 2010).Performance measurements, as continuous improvement tools, monitor and improve actions on a continuous basis; therefore, performance measurement systems assist by identifying areas in need of improvement, diagnosing low performance, planning and implementing changes, monitoring performance to compare results, and developing control systems (Bititci & Nudurupati, undated).As improvement is always possible, the PDCA (plan, do, check, act) methodology results in an effective methodology in the continuous improvement process when assisted by the use of performance indicators (Fortuin, 1988).In fact, a primordial component in continuous improvement is defining the performance indicators and their relationships (Bititci & Nudurupati, undated).
Performance measurement, as an integral part of continual performance improvement (CPI), can be seen from different perspectives: it may be a process to accomplish goals and objectives (Nanni et al., 1990), a process that serves as an agent of change (Brignall, 1991), a process of implementing strategies (Fitzgerald, 1991;Neely, 1998), or a quantification of the effectiveness and efficiency of past actions (Neely, 1998(Neely, , 1995)).Furthermore, Jorgensen et al. (2003) interpret CPI as the implementation of small changes to work processes executed by every member of the organization, making everybody accountable.Nowadays, organizations are more dynamic and willing to adapt to internal and external change; the continuous improvement process assists the organizations in long-term plans for success (Hu et al., 2012).Frequently, sustainability is seen as a process to improve performance; however, it is not expected to achieve the ultimate goal of sustainable performance in a short period of time.It takes time, effort, and commitment; therefore, it is a continuous improvement process frequently assisted by innovation and long-term plans that are periodically measured and compared against pre-established targets.Jaca et al. (2012) state that the concepts of sustainability and sustainable performance are embedded in the continuous improvement philosophy, and the authors add a number of sustainability factors for continuous improvement: management commitment and involvement, key performance indicators linked to obtained results, improvement program objectives linked to strategic goals, the achievement and implementation of results, the use of appropriate methodologies, the assignment of specific resources to improvement programs, the involvement of a task force in the improvement program, adequate training, communication of program results to the rest of the organization, getting more people involved, promoting teamwork, providing a facilitator to support the program, selecting the appropriate area of improvement, adapting to the environmental changes, and recognizing or rewarding participants.Moreover, CPI, sustainable development, and performance measurement are instruments for assessing the sustainability of projects with extended life cycles.To measure sustainability at certain points in time, sustainable development theory and fundamentals and performance measurement principles offer a simplistic snapshot; however, for industries with projects extended for several years, such as surface mining operations in the oil & gas industry, the CPI principles are to be included in the design of indicators and metrics in order to (1) allow organizations to merge their policies, programs, and plans (PPP) in the performance measurement system; and (2) offer the stakeholders an effortless and understandable assessment methodology in which their needs have been attended.

SDIs, KPIs and Metrics
Individuals and organizations are affected by metrics impacting their actions and decisions; behavior is influenced and strategies are created and/or evaluated against them (Hauser & Katz, 1998).Metrics are relevant for an organization to measure progress, but selecting the right metrics is critical for success.Some metrics are easy and simple to measure, but pose counter-productive consequences, while others focus on measuring actions critical for success independently of difficulties and obstacles encountered in the measuring process.Identifying the different KPIs is only part of the continuous improvement process, as any improvement is achieved through continuous planning, monitoring, and execution (Cai et al., 2009).Beck and Oliver (2004) indicate that KPIs drive behavior and are catalysts for success; therefore, measurements are needed to manage strategies, execute initiatives, and evaluate performance.Additionally, KPIs are commonly used to identify work progress improvement areas aligned with objectives; however, the objectives are to be understood as the first step in the continuous improvement process.Thus, KPIs are formulated based on specific targets and objectives.Rosam and Peddle (2003) state that a direct alignment must occur between KPIs and the process or system to be measured, and this must be expressed as a metric; thus, information alone does not possess much benefit for the organization or stakeholders.Additionally, meaningless metrics and a lack of connections between the metrics and goals and objectives do not assist the organization when evaluating resilience (Dalziell & McManus, 2004).
From the customer's perspective, it is critical for organizations to create metrics to measure customer satisfaction while keeping the internal business process and staying profitable and competitive in the business arena.Thus, the organization needs to monitor the changes in its business environment all the time.Stakeholders are customers with different interests in the organization`s operations; therefore, businesses have developed measures for efficiency, effectiveness, quality, and other factors associated with profitable strategies.However, different groups of stakeholders continuously pressure the organizations to develop measures that reflect their sustainable performance.Furthermore, the needs of different stakeholders drive the design of KPIs at the system level (Rosam & Peddle, 2003).In the past, net income, earning per share (EPS), and return on investment (ROI)-among other financial measures-were used to monitor and reward performance (Srivastava et al., 2001).Even though financial performance offers an understanding of the organization's business accomplishments, these figures are not made available in a timely manner throughout the process, and therefore do not address all stakeholders' needs and concerns in other areas (e.g., environment and social).
Sustainable Development Indicators (SDIs) are key performance indicators (KPIs) for sustainability.They are also known as sustainable development key performance indicators (SD-KPIs) or sustainability performance indicators (SPIs), and are used in this manuscript interchangeably.Mistakenly, KPIs are referred to as targets when in fact they are metrics, though Eckerson (2009) points out that the only difference between KPIs and metrics relies on KPIs embodying strategic objectives and measuring performance against goals.Systems' or processes' performances are measured using KPIs; therefore, the relationship between sustainability and KPIs relies on measuring the performance of organizations or projects towards accomplishing the goals and objectives in the social, environmental, and economic areas.External and internal reasons motive the development of SD-KPIs, including stakeholder demands, stakeholder expectations, evolving regulations, and strategic organizational efforts; however, sustainability performance management is still in its infancy (Deloitte, 2012).Macro-level indicators do not contribute to accomplishing the ultimate goals of sustainability; therefore, the areas of impact and potential opportunities must be measured at the micro-(project-) level, linking clearly-defined targets and benchmarks with various levels of the organization.Such links are used to implement strategies through KPIs (Hřebíček et al., 2007); however, initiatives regarding KPIs for SD are focused on the national, regional, and community levels, as the main forces behind SD have been the UN and national governments (Labuschagne et al., 2005).Moreover, developing KPIs for measuring performance at the micro level presents benefits to organizations and their stakeholders.Nevertheless, additional challenges can be encountered, as incorporating the development of effective KPIs for the performance management of the organization's strategic plan is still considered a new concept (Eckerson, 2009).Additionally, SDIs and criteria for sustainability encounter difficulties for proper assessment: (1) when going forth, macro-to micro-level setting objectives must address the question of "what do we want to measure," taking into consideration stakeholders' needs; (2) agreed-upon and multidisciplinary measurements in assessment methodologies not only consider all pillars of sustainability but also allow benchmarking; and (3) the quality of data or metrology increases credibility in the assessment methodology and results.Not only the characteristics of SDIs and indicators define the usefulness of the assessment tool, but improving the difficulties in the assessment methodology assists in the evolution of decision-making processes (Bertrand-Krajewski et al., 2000).Ugwo and Haupt (2007) call for transformations in the evaluation of the sustainability of projects, including hierarchical change in the definition of sustainable development to operational decision-making variables, quantitative and vis-à-vis integrated holistic approaches, and a wider sustainable development agenda.Though current sustainability assessment methods mainly focus on the environmental pillar, and attention is centered on the assessment of buildings (Ugwo & Haupt, 2007), the development of sustainable assessment methods in the last few years is assisting or "pressuring" industry and practitioners to demonstrate sustainability performance (Alwaer & Clements-Croome, 2010).However, without proper assessment tools, no industry-including oil & gas-can demonstrate how well (or how poorly) they are performing vis-à-vis "sustainability".Although SDIs or sustainability KPIs may be assessed using different existing methods-credit-based scoring systems, scaled scoring, comparisons with benchmark or other available options, using credit systems, and involving subjective marking (Ugwu & Haupt, 2007)-the outcomes of the assessment are basically of three types: quantitative, qualitative, or a combination of the two.An outcome is used to analyze the state or progress towards pre-established goals and objectives.
Generic sets of SD-KPIs have been developed, but their usefulness is limited: the applicability of SDIs for different industries demands industry-specific sets of indicators.Therefore, a globally-accepted set of SD-KPIs is a recognizable challenge that is still evolving under the continuous improvement process (Searcy et al., 2005).Agreement regarding SDIs is still under debate, and benchmarking performance across industries or even within the same industry requires the development of specific KPIs for each process in which the tool is meant to be utilized.Primarily, a prerequisite to addressing sustainability includes developing indicators through effective stakeholder participation (Ugwo & Haupt, 2007).Certain characteristics in the design of performance measurement, including SD-KPIs, are to be considered to adequately meet the original intent: organizations and stakeholders should define if their aim is (1) precision or accuracy, (2) positional measurement or directional measurement, and (3) intended or unintended consequences with the measure (Tar, 1995).However, the set of SD-KPIs must remain available to the constant change that occurs in the business environment or due to stakeholder demands.
In a harmonious, sustainable construction environment of industrial projects, a series of key elements is expected in the process of developing effective strategies: the setting of clear and specific objectives, the identification and evaluation of alternatives, and the implementation of selected alternatives (Ugwo & Haupt, 2007).KPIs allow organizations and stakeholders to follow up performance and compare them with pre-established strategic goals and objectives; tracking KPIs' measures is essential in the continuous improvement process (Rosam & Peddle, 2003).Since KPIs are the primordial part of any asset performance management (ASM), the objectives must match measures used; however, difficulties can be encountered: impact of the measures, accountability, processes required to achieve the targets, frequency of measures, data review, and usefulness (Beck & Oliver, 2004).KPIs are characterized as trendable, observable, reliable, measurable, and specific (Beck & Oliver, 2004).Independent of the characteristics intrinsic in SDIs, KPIs, and metrics, the first step in their design is expected to address the stakeholders' needs and concerns; furthermore, in sustainable development and in any decision-making processes, the early involvement of stakeholders defines the level of success in the implementation phase.Additionally, instead of starting with strategies, the design of any performance measurement should address the stakeholders, as they are the process starting point (Neely & Adams, 2005).

SDIs for Surface Mining Operations in the Oil Sands Projects
As the use of indicators in performance measurement makes visible the attributes of the processes (Koskela, 1992), the development and implementation of a set of SDIs for surface mining operations must count with transparency, simplicity, benchmarking ability, effective stakeholder engagement, relatively short cycle time (from data collection to distribution of results) to assist the decision-making process and adjustment capability in a timely manner.Fiksel et al. (2002) indicates that good SDIs measure resource consumption and/or value creation over the product life cycle; therefore, the pre-selected SDIs shown in Table 1 include measures to support the decision-making process in every phase of the projects' life cycle.Additionally, the metrics proposed present a set of diverse characteristics.These include those presented by Tarr (1996): precision versus accuracy, static versus vector measures, soft versus hard measures, and intended versus unintended consequences.There is not a preferred number of indicators to use.A large number of indicators may give the illusion of precision, while in reality, they can limit comprehension and limit the relevance of each (Lee & Burnett, 2006).However, indicators should provide a measure of current performance and assist in setting targets for what might be achieved in the future (Jefferson et al., 2007).The pre-selection process utilized for determining the SDIs for surface mining operations resulted with four hierarchical categories of indicators: prerequisite (mandatory) indicators, desired indicators, inspired indicators, and non-active or non-applicable indicators.These four categories are also identified in the Design Quality Indicator framework (Design Quality Indicator Framework , Undated).If indicators are to be classified between qualitative and quantitative, they are equality important (Eckerson, 2009).While some are based on quantitative data, others are based on qualitative or subjective data, which is obtained through data collection processes.This presents the additional benefit of engaging different stakeholders in the process.
Environmental and social responsibility are increasingly influencing decision-making and business strategies; therefore, practices for SD evaluation and assessment are gaining ground in all industries, including in those called "too big to fail".However, the oil & gas industry (including surface mining operations) is affected by the same global uncertainty found regarding the ramifications of pursuing the implementation of sustainable development (Berkel et al., 2008).When it comes to SDIs, social and economic indicators are the least developed (Fiksel et al., 2002).The debate relies on two main questions: what indicators should be measured (i.e., what indicators should be included) and how those indicators should be measured (i.e., which metrics are appropriate to indicate quality of life and community prosperity).Therefore, it is valid to distinguish between indicators and metrics: the first define what is to be measured and the second define how it will be measured.Moreover, trade-offs are an additional problem with performance metrics for which decision-making tools are being used.An option for decision-making tools is to weight each KPI using the AHP.Although the process assists by determining the weight of each KPI, it does not specify the relationship amongst them (Cia, 2009).A good starting point is the balance scorecard (BSC), which was developed with the aim of integrating non-monetary, qualitative, and "soft" issues related to social and environmental factors (Bieker & Waxenberger, 2002;Figge et al., 2001Figge et al., , 2002aFigge et al., , 2002b;;Schaltegger & Wagner, 2006).Schaltegger and Lüdeke-Freund (2011) take further the concept of BSC to refer to an integrated sustainable performance measurement, the sustainability balanced scorecard (SBSC), in which the critical factor of integrating financial, non-financial, quantitative, and qualitative information is addressed.
Types of indicators, selection of criteria, and characteristics of the indicators for a set of performance measurements are commonly debated, and extensive literature can be found; however, for performance measurement in an implemented continuous improvement process, the KPIs proposed are quantifiable, meaningful (as the result of being based on available and reliable dates), relevant to the needs of stakeholders, aligned with the chosen strategies, a manageable number (i.e., not too numerous), and comparable over time (Bouchery et al., 2010).Fiksel et al. (2002) suggest a five-step distinctive process when selecting KPIs or SD-KPIS: (1) consider stakeholder needs, (2) identify important aspects, (3) establish company goals and KPIs, (4) select performance indicators and metrics, and (5) set targets and track performance.Within the selection process, certain criteria assist in choosing the appropriate KPIs; as suggested by Hřebíček et al. (2007) and applied to those shown in Table 1, KPIs should give an accurate appraisal of the organization's performance; be understandable and unambiguous; allow for year-to-year comparisons; allow for comparisons with sector, national, or regional benchmarks; and allow for comparisons with regulatory requirements.
As improvement needs to be continuously sought and monitored according to the continuous improvement definition (Dale, 1996), so must the SDIs and metrics for surface mining operations in the oil sands projects.From the organization's point of view, a certain level of investment is required to implement a measuring performance system, but a number of benefits are expected in return when managing and reporting performance: cost, productivity, market advantages, image and reputation, and employment recruitment, among others (Defra, 2006).Bourne et al. (2000) propose three phases in the design of performance management systems: design of the performance measures, implementation of the performance measures, and use of the performance measures.Even though the literature is dominated by processes that answer the question "what should we measure?" for certain areas, for sustainability performance, the answer is still under ardent debate.However, in the design phases, as suggested by Bourne et al. (2000), two requirements involve identifying the key objectives to be measured and designing the measures.Nevertheless, there is an absence of agreed-upon indicators and metrics, and high uncertainty remains regarding decisions on sustainable technology (Berkel et al., 2008).Since performance measurement systems use a set of metrics or individual performance measures to quantify efficiency and effectiveness (Neely et al., 1995), a sustainability rating system-used as a performance measurement system-can be the crucial tool to link macro-with micro-or project-oriented objectives.Developing and implementing a sustainability rating system for surface mining operations is a relatively difficult task for additional reasons, such as the fact that the oil & gas industry is project-oriented, and projects are usually quite complex; however, since people drive their attention to what can be measured (Waggoner et al., 1999), a performance measurement system in the form of a rating system cannot be overlooked, and instead, its usefulness can be optimized.

Performance Improvement Factors (PIF)
The proposed integrated approach to sustainability assessment consists of three distinctive areas: continuous performance improvement (CPI), sustainability theory and principles, and multi-criteria decision-making.The performance improvement factors (PIFs) for the sustainable development indicators (SDIs) metrics for surface mining operations in the oil sands projects are designed in alignment with the principles of the CPI process; therefore, any performance improvement is reflected in the PIF value for each indicator.The PIF will demonstrate the progress of the improvement process.As pointed out by Dale (1996), the progress potentially shows changes in behavior and attitude; improvements in operational and business performance indicators; and the alignment between the company's strategies, policies, and guidelines and projects' quality improvements outcomes.
In performance management, a key principle consists of measuring what you can manage (Weber & Thomas, 2005); however, it is commonly found that even factors that can be managed are because of the subjectivity encompassed in most factors.Assessment methods that are already developed assist by putting subjective matters into measurable metrics that are easy for stakeholders to comprehend; nevertheless, sustainability assessment still has challenges to overcome.Moreover, the assessment of all pillars of sustainability has not been developed evenly.In fact, literature often refers to social issues as the most under-developed pillar, and the economic issues not far ahead.Interestingly, environmental factors are the center of most assessment methods.This focus may have been influenced by stakeholders, who often associate sustainability with only the environmental impacts of projects and organizations, leaving behind social and economic factors inherent in the fundamentals of sustainable development.Additionally, environmental issues are easier to identify, their impacts more noticeable, and most stakeholders believe they have the right to express their opinions based on idealistic knowledge that most lack.
The measurement of social, economic, and environmental impacts faces additional challenges as a consequence of the subjectivity factor; therefore, linking subjective with tangible (objective) measures presents a valuable alternative.In fact, different assessment methodologies measure impacts in terms of monetary value (a common utilized metric unit).While environmental and economic impacts can mostly be measured using different existing assessment methodologies and/or are directly linked to economic parameters, social impacts assessment methodologies still encounter difficulties in putting value on undefined but notable impacts.A truly effective rating system designed to measure sustainable development must include the three pillars (i.e., social, economic, and environmental) of sustainability.The objective of a rating system is to present the different stakeholders with an overall score instead of segregating each pillar of sustainability; nevertheless, each indicator must have the characteristic of being able to be measured separately, so the users easily identify the roots of each indicator metric and organizations can focus on areas needing some improvement.Once indicators and their metrics have been defined, the improvement (negative or positive) over time needs to be measured.Poveda andLipsett (2011a, 2011b) propose the WA-PA-SU project sustainability rating system to measure the sustainability performance of projects and organizations.The assessment methodology utilizes the performance improvement factor (PIF), defines as: "a factor to determine the degree of negative or positive improvement of a specific criteria (i.e., indicators) during a specific period of time" or, PIF = Indicator Performance Actual Value (Metric) / Indicator threshold or baseline Value (Metric Baseline) [1] The aim of measuring the PIF is to move the focus from the specific value of an indicator(s) to the continual performance improvement process.As indicators can either (1) measure the closeness to a defined target, which is the most commonly used approach, or (2) give the direction of an indicator so the others move in the desired direction, thereby motivating change and performing better over time, the PIF methodology combines both approaches by setting initial baselines but awarding higher scores as the PIF value increases.This motivates projects and organizations to focus on the overall continual improvement process.The PIF is designed with the aim of allowing organizations to not only assess their current performance, but also improve throughout time or the project duration by identifying the indicators that have PIF values that fall below 1. PIFs below the baseline value of 1 are under-performing.In some cases, the baseline may be easier to identify, as they can be imposed by federal, provincial, or local authorities through regulations; however, a value given by a regulation does not guarantee sustainable performance.The baseline of other indicators needs to be created through stakeholder consultation, scientific support, or simply by setting an initial arbitrary threshold with the intent of improving it once additional research is done or stakeholder consensus occurs.
Three types of PIFs are identifiable in the methodology, based on which metrics are used to measure indicators:  Type I: based on relevance factor measurement (i.e., relevance factor or subjective stakeholder valuation)  Type II: based on performance improvement (i.e., comparative assessment method)  Type III: based on level of investment (i.e., link to economic metrics) The continual performance improvement (CPI) indicator measurements can be achieved by using three different categories of assessment methodologies: (1) quality and performance audits, (2) performance improvement measurements, and (3) impact vs. investment ratio improvement; henceforward, these three different alternatives are proposed to assess the performance of indicators (i.e., metrics), which are typically linked to type I, II, and III PIFs, respectively.

Relevance Factor or Subjective Stakeholder Valuation
For certain indicators, it is frequently found to be difficult to determine their metrics, which adds to the level of uncertainty surrounding their assessment; therefore, alleviating the potential inherent subjectivity becomes a priority.The relevance factor or subjective stakeholder valuation proposes the involvement of stakeholders and the linking of subjective valuations to objectives' measures.In sustainable development, as in any multi-criteria decision-making process, the effective involvement of stakeholders determines the success of the process, due to the accountability factor desired for some and avoided by others.Additionally, objectives measures are recognizable and frequently used by most stakeholders, and measuring them is economically viable with data collection procedures already in place.
To determine the PIF type I, which is based on the relevance factor measurement, it is necessary to define the methodology to calculate at any given time the performance value of a certain indicator that follows under this category; however, a direct indicator performance value is either linked to an objective measure or a 9-point semantic differential scale, as shown in Graph 1 and 2. In the rating system, users have three different options to calculate the PIF:  Determining the relevance category (stakeholders' input) linked with the project's or organization's energy consumption (Graph 1)  Determining the relevance category (stakeholders' input) linked with the project's or organization's greenhouse gases (GHG) generated (Graph 1)  Determining the relevance category (stakeholders' input) linked with the organization's or project's bbl/d or bpd production (Graph 2) Organizations or projects may eventually have information to calculate the PIF of an indicator using more than one option.The intended continual improvement approach suggests using the lowest PIF calculated.Lower PIF values translate into higher levels of energy consumption, greenhouse gases generated, or extraction of non-renewable resources (i.e., oil, gas), as well as a lower number of points obtained in the overall sustainability rating.
Due to limitations found in determining the performance value of certain indicators (metrics), the PIF offers an alternative to calculate the overall performance of an organization or project.Instead of focusing on a particular indicator metric, stakeholders' and objectives' measures link the indicator to more sustainable goals, such as lowering the extraction of non-renewable resources, the production of greenhouse gases, and/or energy consumption.Moreover, instead of focusing on an indicator's performance value, the interest lies in the improvement of the indicator over time; therefore, the PIF is not a metric, but instead, a factor.
The PIF value for indicators in this category ranges from zero (0) to two (2).For indicators under PIF type II and III, once a baseline is determined, the performance improvement is measured by comparing the actual performance against the threshold (see Formula 1).An indicator obtains a PIF value of one (1) if it meets the indicator's pre-established baseline; as difficulty arises establishing baselines or thresholds for some indicators, the first measurement serves this purpose.Organizations' and projects' vision regarding sustainability should move towards eliminating energy consumption, greenhouse gases generated, and/or the extraction of non-renewable resources; therefore, a PIF value of two (2) is awarded.As noted in Graphs 1 and 2, stakeholders assign the relevance category for each indicator.Some may indicate that even with zero energy consumption or generation of greenhouse gases and the elimination of the extraction of non-renewable resources, the organization or project is not able to obtain a PIF value of two (2).The purpose for users, stakeholders, and developers is to eliminate the activity described by the indicator-or at least minimize its impact-to the point that stakeholders consider the indicator relevance category in the range of low-low.As a consequence, a PIF of two (2) points can be awarded.

Comparative Assessment Methods (CAM)
The second and most common group of metrics for indicators is conformed for those linking the impact directly to a measurable aspect of the organization or project (e.g., CO 2 /per ton of bitumen produced, trees harvested/acre).The quantitative characteristics of these metrics serve as an advantage for projects and organizations to report their performance to different stakeholders, though measurability (in some cases) may limit the data collection, which may force a certain level of investment.In the mix of indicators, economic metrics tend to measure at the national or global level (also known as macro-level, e.g., gross national product [GNP], gross domestic product [GDP]) instead of focusing at the project and organization levels (i.e., micro-levels).In fact, some argue that economic indicators in their current form are not even meaningful measures of economic sustainability (Sheng, 1995).Subjectivity is intrinsic in metrics associated with social indicators; questionnaires with graded scales are usually used to measure perceptions and feelings connected to standards of living or quality of life.The environmental indicators measure the impact in three contexts; air, land, and water.Fortunately, progress has been made by regulatory agencies to set thresholds (i.e., limits) for projects or organizations to reduce the impacts of their operations.Indisputably, stakeholders drive the design of most indicators and metrics.The "social license" to operate is the main aim of projects and organizations, as concerns about corporate environmental and social responsibility has gradually increased.Environmental metrics are far ahead in their design, driven by public perceptions of sustainability, investment of research, and mainly, the creation of limitations by regulatory agencies.However, social and economic performances are less supported by regulations, which tend to be weakly developed and poorly enforced.Measurability and the characteristics of the metrics for each pillar of sustainability imply that comparative assessment methods are mostly used by environmental indicators and, to a much smaller degree, for economic indicators, leaving social impacts, for the most part, to subjective stakeholder valuation (i.e., relevance factor) or linked to economic metrics methodologies.
Simplicity is the main objective of an indicator, and while stakeholders search for it, decision-makers demand indicators that can be integrated into the relevant levels of the policy-making process; however, environmental and social indicators are brought into the economic policy-making arena without decision-makers knowing their actual monetary value.Therefore, indicators that are linked to policy targets and compatible with macro-economic indicators and the budgeting process are often preferred.Conversely, not all indicators included in this category (i.e., comparative assessment methods) are designed with the support of the policy-making process (i.e., regulatory agencies); some metrics indicators are designed to set a threshold (i.e., baseline) for measuring the impact, making use of practicability and simplicity.For example, the number of trees harvested per acre illustrates such characteristics.
Metrics for indicators designed using comparative assessment methods not only are often linked to the policy-making process, but also meet the simplicity objective at different levels: measurement, reporting, and stakeholders' understanding of the indicators used.The difficulty arises when designing a sustainability index, since each indicator uses different metrics; therefore, multi-criteria decision-making is frequently used to decide the weights of each indicator.However, from the continual performance improvement (CPI) point of view, the main focus relies on measuring the improvement of each indicator in a specific period of time.The PIF measurement for indicators using comparative assessment methods utilizes Formula 1.As a result, PIF type II is calculated in a straightforward manner to measure the performance improvement of those indicators.

Link to Economic Metrics
The third group of metrics for indicators proposed to calculate the PIF type III is based on links to existing and recognizable economic metrics (e.g., level of investment per return of investment [ROI]).Nowadays, projects and organizations often budget or set aside a certain amount of the total cost of projects and/or operations for elements such as quality, risk, and safety; the level of investment (e.g., budget) has increased as organizations became aware of the different inherent benefits.Additionally, the pressure from different groups of stakeholders increased through the years; therefore, organizations found an additional justification for their monetary increment to improve quality, risk, and safety performance.Similarly, as sustainable development is still in its early stages of implementation, which includes education and awareness of the public and stakeholders, the level of investment is expected to increase to (1) mitigate social, economic, and environmental impacts; (2) promote sustainability; and (3) implement sustainable designs and practices.Therefore, designing metrics for indicators that link the impact with the level of investment indicators offer to stakeholders, organizations, and policymakers results in a certain level of confidence in more sustainable operations and a step in the right direction for a compromise in the areas of social and economic responsibility and environmental performance.
In the same way the PIF type II is calculated, PIF type III is a straightforward implementation of Formula 1. PIF type III not only presents the performance improvement value of a specific indicator, but also the continuous commitment by the projects and organizations to sustainability; a higher PIF type III value indicates a higher investment to mitigate social, economic, and/or environmental impacts.Additionally, the PIF type III measurement assists practitioners in benchmarking their performance.Instead of associating their performance to a certain amount of investment independent of the project's and/or organization's size, the PIF value compares the ratio of investment amount of projects and/or organizations, putting everybody on similar parameters of assessment.
PIF type III is not exclusively designed for economic indicators.In fact, social and environmental indicators may benefit using metrics that associate the level of investment of projects and/or organizations in mitigating different impacts.a profit while investing in their work-force-health, safety, and staff development; sustainable development of their surroundings (i.e., nearby population centers); and sustainable development initiatives that benefit their stakeholders (e.g., innovation, education).Nevertheless, internal economic indicators (e.g., return of investment) and global indicators (e.g., inflation rate) may be a reflection of a healthy economy for organizations and the government; therefore, they are indicative of sustainability.However, the size of the project, organization, or industry is relevant to measure the contribution of factors such as inflation rate, GDP, and GNP, since those are not a metric of a specific industry, but instead are aggregates of the economic performance of a variety of factors (i.e., industries or sectors).

Conclusions and Future Research
It is commonly said that we are what we measure; nowadays, with the introduction of sustainability and sustainable practices becoming more popular, we measure what we want to become.This manuscript introduces part I of a new assessment methodology for sustainability of industrial projects, with an emphasis on oil sands for surface mining operations; the concept of the performance improvement factor (PIF) based on the continual performance improvement (CPI) offers to organizations and their stakeholders a tool to demonstrate their commitment to sustainable practices throughout the time, allowing adjustments in those areas in which the PIF falls below 1. (Indicators with a PIF below 1 point out performance below the baseline or threshold) Additionally, the PIF presents all indicators to stakeholders as a factor of improvement instead of a different metric for each indicator.This contributes to alleviating the commonly-found confusion factor.Assessment methods present their results for each of the indicators of sustainability; therefore, the stakeholder must get familiar with each metric to understand the assessment.After measuring each indicator's performance, the proposed assessment methodology (i.e., the sustainability rating system [WA-PA-SU Project Sustainability rating system]) presents the improvement of each indicator by measuring its performance against itself in a determined period of time.Conversely, the stakeholders are not to drive their attention to how the indicator is measured, but instead on its improvement over time.
CPI is a critical factor for projects whose operations have a long execution phase, such as surface mining.While some factors can be forecasted and may be controlled or mitigated, others may be unforeseen.Implementing measurement systems (indicators) for sustainability has mostly been focused in projects where most sustainable practices can be implemented during the planning and execution phase (e.g., the building industry), with the expectation of an optimum performance during the operation phase so the project and organizations obtain the expected score in the pre-selected rating system (e.g., BREEAM, LEED, CASBEE) to measure their sustainability performance.The surface mining industry experiences an amalgamation of the execution and operation phases; therefore, most indicators for sustainability are designed to be measured primarily during the operation phase of the projects, instead of during their execution (i.e., construction).Moreover, the indicators for sustainability should reflect the projects' and organizations' performance over long periods of time instead of over a short construction phase period (as occurs in the building industry).
While the PIF is one of the three components in the proposed integrated approach for sustainability assessment, the stakeholders' assessment of indicators and their weights becomes essential for the calculation of the overall rating of sustainability performance.Stakeholder engagement in multi-criteria decision-making processes, including those for sustainability assessment, assists in the design of rating system methodologies by adding the credibility factor and facilitates the implementation of the rating system; therefore, future research requires the assessment of the indicators' weights, and further developments are expected to include the integration of the multi-criteria decision-makings process results with the PIF.Additionally, the integrated methodology for sustainability assessment requires further development, including case studies to test the methodology and evaluate its applicability, usefulness, and implementation cost.

Table 1 .
Pre-selected SDIs for surface mining operation in oil sands projects 1Mining effluents include: arsenic, cooper, cyanide, lead, nickel, zinc, total suspended solids, radium, pH 2 GHGs include: sulphur dioxide (SO 2 ), ozone (O 3 ), nitrogen Dioxide (NO 2 ), particulate matter (PM2.5),carbon monoxide (CO), oxides of nitrogen (NOx), volatile organic compounds (VOCs), hydrogen sulphide (H 2 S) not measured, due to factors such as cost, applicability, and/or usefulness.Additionally, the control factor influences the manageability of what is measured; as Peter Drucker states, "It is not possible to manage what you cannot control, and you cannot control what you cannot measure."Conversely, a main challenge in sustainability consists of defining the indicators (i.e., what to measure) and the metrics (i.e., how to measure the indicators) for all the pillars of sustainability.A paradox linking sustainability, performance improvement, and continual performance improvement indicates that while trying to manage organizations and projects towards more sustainable results, there is a lack of agreement amongst indicators and metrics to measure any given organization's or project's performance.Therefore, managing, controlling, and determining improvement of indicators and metrics that have not been determined or agreed upon becomes a matter of defining baselines to then improve those measurement factors through research, practice (i.e., experience), benchmarking, and stakeholder engagement, among other alternatives.Certainly, in measuring each area in sustainable development (social, economic, and environmental), challenges are encountered mainly

.91 1.03 1.16 1.28 1.41 1.53 1.63 1.78 1.78 1.63 1.53 1.41 1.28 1.16 1.03 0.91 0.78
Controversially, economic performance indicators have often been used as economic indicators for sustainability.Using the PIF type III assists in linking the macro-level (i.e., global) objective of sustainability with the micro-level (i.e., project or organization) by connecting the sustainable development of industry to factors such as employment and investment in human capital.Indisputably, organizations require a level of profit to survive.However, rather than measuring how wealthy an organization has become in certain period of time, or the contribution of a project to the organization's capital as an indicator of sustainability, organizations should demonstrate their capability (i.e., economic sustainability or sustainable survival) to create