Proposal and Implementation of a User-Centered Design Approach for Smart Grid Technologies: Example of the VERTPOM® Project

Acting to preserve our planet as much as possible is no longer optional in today's world. To do so, Smart Grids within the framework of electrical networks involving not only Distribution System Operators (DSOs), but also consumers in their Energy Demand Management (EDM) activity represent an innovative and sustainable solution. However, the integration of Smart Grids into network management or into consumers' homes implies changes at several levels: organizational, social, psychological, etc. This is why it is essential to consider the human factor in the design of the technologies used in these Smart Grids. This paper proposes the integration of DSO operators and consumers within a user-centered evaluation approach in order to design Smart Grids that are sufficiently acceptable to users to enable Positive Energy Territories that produce more energy than they consume. This demonstration will be illustrated by the VERTPOM® project aiming at facilitating the use of renewable energies specific to each territory in order to contribute to the reduction of greenhouse gases and make the territories less dependent on traditional energies, and thus make Picardy (in France) a Positive Energy Territory. This paper presents the user-centered evaluation approach applied to three technologies (i.e., the VERTPOM-BANK® supervision tool intended for DSO operators, the private web portal and the IBox smart meter intended for households) from the upstream design phase to the implementation of the technologies in real-life situations.


Introduction
The Commission de Régulation de l'Energie (CRE) defines Smart Grids as "an energy network that integrates information and communication technologies, contributing to an improvement in its operation and to the development of new uses such as auto-consumption, the electric vehicle or storage". The European Commission defines it as "an electricity network that can intelligently integrate the behavior and actions of all users connected to it -producers, consumers and those who do both -in order to efficiently provide a sustainable, economic and secure supply of electricity". For users to have optimal management of their energy and water consumption, Smart Grids must match the Energy Demand Management (EDM) practices of consumers. The quality of users' consumption management will depend on their experience with these technologies: for them to benefit, it is essential that they perceive these systems as a way of achieving their final objectives.
The educational website Connaissance Des Energies (CDE), whose content is validated by a Committee of Experts, traces the origins of Smart Grids as follows: The first stage of their emergence was the introduction of the automatic meter reading function in the 1980s. About ten years later, the concept of the communicating meter was born and contributed to the development of a first example of Smart Grid, in the framework of the Telegestore project in 2000. At the same time, in the United States, the first measuring system with fast, long-distance sensors became operational. The term Smart Grid was generalized in 2005 thanks to the "Smart Grids" technological platform that was set up by the European Commission. Today, Smart Grids are gradually developing, and the adaptation of infrastructures takes time. Smart Grids are the subject of many projects such as Smart Occitania (2017-2020), SMILE (2017-2020) and many others. Jansen et al (2020) identified 75 laboratories conducting research on Smart Grids, of which 58 are located in Europe. Also according to these authors, 91% of the laboratories identified are conducting research in the field of production and distribution of energy resources, and 55 laboratories are conducting parallel research on energy storage. In view of these figures, there would appear to be a strong interest in these technologies insofar as the knowledge generated by this research suggests numerous prospects for Smart Grids.
Smart Grids are intended to be used by different profiles of people, especially since they are intended to assist Distribution System Operators (DSOs) in their work activity, and consumers in their DSM activity. It seems more than necessary to put the end-users at the center of the design, and as early as possible in order to "design for them and with them" (Lallemand & Gronier, 2018). The user-centered design approach, as defined in ISO 9241-210 (2010), involves performing four tasks. The first task is to understand and specify the context of use through an analysis of the environment in which the tool will be integrated. The second task is to analyze user requirements. The third task is related to the production of design solutions (eg, proofs of concept, prototypes) on the basis of technical knowledge on the one hand, and knowledge about users and the context of use on the other hand. Finally, the fourth task concerns the evaluation of these solutions. It should be noted that this process is iterative until the desired result is obtained. In other words, intermediate solutions (i.e., prototypes) must be tested and their performance improved so as to obtain an optimal final solution.
While previously, User Centered Design (UCD) focused on the needs of users and their activities (Norman and Draper, 1986) and sought to guarantee the usability of systems, the formalization of the ISO 9241-210 standard (2010) now also includes the notion of user experience or User eXperience (UX) in this approach, thus recognizing the importance of emotions. The UCD is a framework for designing useful, usable technologies that promote a good UX (Loup-Escande, et al. 2013), ensuring better acceptance of these technologies (Loup-Escande and Loup, 2020). Burkhardt and Sperandio (2004) define utility "as a significant benefit or amenity of the artifact to the user in activities mediated by a computer system, this benefit always being relative: to user goals; to existing or habitually used tools; to the environment of use; to dependencies with other activities. " (Loup-Escande, 2013, p.5). According to ISO 9241-11 (1998), usability is "The degree to which a product can be used, by identified users, to achieve defined goals with effectiveness, efficiency, and satisfaction, within a specified context of use" (p.2). Hassenzahl and Tractinsky (2006) define UX as "a consequence of a user's internal state (predispositions, expectations, needs, motivation, mood, etc.), the characteristics of the designed system (e.g., complexity, purpose, usability, functionality, etc.) and the context (or the environment) within which the interaction occurs (e.g., organisational/social setting, meaningfulness of the activity, voluntariness of use, etc.)" (p. 95). Arhippainen and Tähti (2003) include social factors (time pressure, success or failure, etc.) and cultural factors (habits, religion, symbol, etc.) This article sets out to propose a user-centered assessment approach for designing useful, usable, UX-enabled and ultimately acceptable Smart Grids, by describing the way it has been implemented and tested in a real-life project called VERTPOM®. The following section presents and explains the specificities of the VERTPOM® project. Section 3 describes in detail the user-centered assessment approach implemented in the framework of this project from the upstream design phase to the implementation of technologies in real-life situations. In conclusion, the results are discussed in relation to previous works referenced in the literature.

The VERTPOM® Project
As explained by Loup-Escande, Capo and Raharison (2020), VERTPOM® (True Energy of the Territorial POsitive and Modular) corresponds to the experience in Picardy of a multi fluid site (electricity, gas and water). The aim of the project is "to facilitate the use of renewable energies specific to each territory, in order to contribute to the reduction of greenhouse gases and make the territories less dependent on traditional energies, while at the same time becoming Positive Energy Territories, producing more energy than they consume" (p. 81). The ergonomic contributions introduce the point of view of energy consumers making them "actors" of the energy transition issues, as well as the issues related to the supervision and management of the energy network in the design cycle.

Objective
The objective of the operator-centered assessment is to guarantee an excellent compatibility between the VERTPOM-BANK® software and the work activity of the operators, thus allowing an optimal performance of the interaction between the human and the system (Jordan, Thomas, Weerdmeester & McClelland, 1996). More precisely, it is not only a question of revealing the axes of improvement, in particular concerning the ease of use (i.e., usability) and the relevance of the functionalities in relation to the work activity, but also of collecting the operators' opinions on the uses of this tool in their work activity (i.e., utility). This is all the more important because, according to Nogier (2008), these dimensions determine operator performance insofar as the latter will perform a task more quickly in a pleasant working atmosphere.

Methods
Two complementary methods were deployed: assessment by ergonomic inspection, which consists of identifying interface elements to be improved (Brangier et al. 2015), and user testing sessions. Ergonomic inspection is an assessment approach based on criteria, such as those proposed by Bastien & Scapin (1993), for the assessment of computer devices. This method is generally used to discover and correct design flaws, before carrying out user tests (Bastien & Scapin, 1993). The result of such an assessment is a set of recommendations based on the representation of a third party who is not the end user. In the case of the VERTPOM® project, this third party was an ergonomist. The data was collected by assessing the application of the Bastien & Scapin (1993) criteria, by taking notes on discussions with the developers, and by recording them in audio and video format.
The user testing of prototypes is a simulation method in the sense of an artificial reproduction of a phenomenon to be studied, specifically aimed at reconstructing a work or use situation in order to observe and induce behaviors close to those likely to appear in a real future situation (Baccino, Bellino & Colombi, 2004;Dow, Heddleston & Klemmer, 2009). This method has several advantages for the design of emerging technologies such as Smart Grids. Based on an observation of the actual impacts of technology on user activities (Bobillier-Chaumon, Rouat, Laneyrie & Cuvillier, 2018), it is possible to identify the conditions that will enable users to get real benefit from their use (Burkhardt & Lubart, 2010). The prototypes are also a means of projecting users into a future use context incorporating the technology being designed (Anastassova, Mégard & Burkhardt, 2007). So, users are able to propose features and properties and thus contribute to enriching practical acceptability by integrating these new needs during design (Newell, Carmichael, Morgan & Dickinson, 2006).
Within the framework of the VERTPOM® project, two operators were involved -namely, the technical director and the distribution network manager of Gazelec de Péronne -in these simulation situations. At the request of the operators, the developers made a complete presentation of the VERTPOM-BANK® software. This presentation also allowed the operators to ask questions, to expose situations they encounter and to simulate them with the developer at the controls first, then by manipulating the software themselves. For organizational reasons, the presentation session and the user test session took place two months apart for each operator. Following these introductory exchanges, the operators completed a pre-questionnaire aimed at collecting socio-demographic data, information on their work tools, their degree of technophilia and their expectations regarding the VERTPOM-BANK® software. Each participant then completed the tasks following three usage scenarios co-constructed with Gazelec de Péronne. The first scenario consisted of updating the network mapping and observing the network behavior to determine if the data seemed coherent, given the elements known by the operators concerning their work activity. The second scenario put them in the context of a storm that caused a tree to fall on a power line: after locating the line, the operators were asked to rebalance the network and perform metering checks. In the third scenario, a kite with a handline in contact with water caused a short circuit on a power line, and the operators were asked to rebalance the network through a series of actions. These runs were videotaped. After the runs, operators then completed a questionnaire consisting of the "AttrakDiff" UX scale (Hassenzahl, 2003;Lallemand & al., 2015) and the "CSUQ" usability scale (Lewis, 1995;Lallemand & Gronier, 2018). In addition to the quantitative data collected through questionnaires, a semi-structured interview allowed the operators to express themselves more freely and to clarify their positioning in relation to their experience with the VERTPOM-BANK® software. Specifically, this interview made it possible to address the usability of the software (Brangier and Barcenilla, 2003) in terms of effectiveness (i.e., "the ability of the interface to do what is expected" (Brangier, 2006, p.4), efficiency (i.e., "the ability to produce a given task with the minimum of effort" (Brangier, 2006, p.4 ) and satisfaction which "refers to the level of comfort perceived by the user when using an interface" (Brangier, 2006, p.5), or even ease of learning and tolerance of errors of the system expressing "the capacity of the interface to remedy the deficiencies, errors and mistakes made by the user, by proposing help, guidance or relevant messages for example" (Brangier, 2006, p.5). This interview also allowed us to collect the operators' point of view on the usefulness and relevance of VERTPOM-BANK® in the context of their work activity. These interviews were recorded for transcription and analysis.

Results: Ergonomic Inspection
The assessment by ergonomic inspection focused on the four main functional sections of the VERTPOM-BANK® software developed par LTI-EESA partner: home, simulation, parameters and optimization.
The "Home" screen is divided into four areas ( Figure 1): 1) The horizontal navigation bar at the top; 2) The main "Operating Diagram" in the center; 3) The equipment list; 4) The presentation of the simulation status. This "Home" section was the subject of 12 recommendations associated with guidance issues (e.g., indicate the manipulations to be done to zoom in on the operating diagram), workload issues (e.g., implement the function enabling the simulation to be configured directly in the "Simulation" section), homogeneity/consistency issues (e.g., using the same format for titles).

Figure 1. "Home" section
The "Simulations" section ( Figure 2) includes functions that allow the user to select the type of simulation (e.g., Snapshot 1 ) the phase of interest (e.g., phase A 2 ), the type of measurement of the graph that will be generated (e.g., Voltage), the percentage of load distribution (e.g., 50%), and the implementation of errors in the simulation (e.g., entering the value for electrical resistance). The latter was the subject of 9 recommendations in terms of guidance (e.g., indicating the procedure to undo actions, go back), workload (e.g., choosing an item in the drop-down menu: offering the possibility of entering values in the field and proposing the items containing the values entered in the drop-down menu), error management (e.g., asking for confirmation of the deletion actions for the "Delete" and "Delete all" buttons), homogeneity/consistency (e.g., using the same terms for labels that refer to the same functions) and meaning of codes and denominations (e.g., changing the title of the "Fault Menu" sub-heading because it is not representative enough of the function and what it is possible to do).

Figure 2. "Simulation" section
The "Settings" section includes two functions that allow the operator to configure the color of the lines according to the results of the simulation ( Figure 3a) and to create counters in order to see what is happening in real time on the simulation in a "Measure Point" pop-up window that exposes the data according to the type of measurement chosen (Figure 3b). This has been the subject of two recommendations related to homogeneity/consistency (e.g., using the same format as in the rest of the interface for the input fields), and workload (e.g., making the "Settings" section a sub-section of "Simulation" because it enables simulations to be configured). The "Optimization" section allows the integration of batteries in the simulation in order to define the best time to charge and discharge the batteries by analyzing the financial benefits that the use of these batteries could bring ( Figure 4). Two recommendations were made regarding guidance (e.g., implementing a progress bar to inform about the processing of the request in the "Optimization" section) and homogeneity/consistency (e.g., clearly differentiating input fields from plain text).  With regard to the methodology directly involving users, the pre-questionnaire made it possible to profile the operators who are also the end users of the VERTPOM-BANK® software. They are male, with an average age of 31, and hold positions of responsibility: one said that his profession had been "GRD technical manager" for 2 years and the other said that he had been the "technical assistant" of the GRD manager for 4 years. They both use a supervision tool for multi-fluid networks, including electricity, which the technical assistant has been using for 3 years. While the technical assistant uses simulation, supervision and geolocation software on a daily basis, the DSO technical manager states that he uses simulation software several times a year (but less than once a month) and supervision and geolocation software several times a week (but not to the point of using them every day). The two operators seem to be quite similar in terms of technophilia: in general, the DSO technical manager being slightly less comfortable or up to date when it comes to new technologies, especially in the recreational sphere, than the technical assistant. However, the GRD technical manager indicates a rapid understanding of how a new technology works when he uses it. The technical assistant, on the other hand, indicates that he is completely at ease with new technologies, both in terms of manipulation and understanding how they work. Both operators say that they are keen to experiment with new technology in their work as soon as they hear about it. Their expectations of the VERTPOM-BANK® software include not only the ability to simulate network operations to optimize the network and undertake the necessary work on it, but also the simplification of their work activity.
During the tests, the operators explored all the functions of the VERTPOM-BANK®, guided by the scenarios. In this regard, eleven areas for improvement were mentioned following this manipulation. For example, they noted that the size of the window was too small and not visible enough, the need to confirm an action via a Play button, or the presence of useless information such as the curve proposed in the "Diagnosis" section, which was redundant with the data present in an existing sub-section called "Meters". In addition, the operators particularly appreciated several aspects implemented in the software, such as the color code enabling a rapid visualization of the state of the network or the measurements providing details on the state of the various system elements. According to the operators, the VERTPOM-BANK® software would allow them to "save time" and "work a little more safely".
During the interviews following the interactions with the software, the operators mentioned some concerns about the implementation of the system in their work activity in terms of hardware (e.g., the GRD technical manager is concerned that a smaller screen would imply a reduction in the size of the operating diagram, thus altering its readability, while the technical assistant commented that organizational issues (e.g., updating the diagram 3 would represent an additional workload, knowing that the technical assistant is already doing it on two other supports) and adding another support could be a factor of non-use of this system ("Well, I already have three PCs on my desk. So it's another application. Certainly it's good, but in that case we should get rid of some of our existing ones and use this instead"). These discussions with the post manipulation operators also allowed us to make some recommendations. For example, it will be necessary to set up the exploitation of the same diagram by the various applications.
The questionnaire responses were an important source of information about the operators' experience with the software. The scores obtained from the CSUQ show that the operators are globally satisfied with the system (score=6/7). Indeed, they not only found the system easy and simple to learn and use, but also that they performed the work activity involving the software efficiently and quickly without having to put in much effort. However, both participants agreed that the information provided by the system was not clear enough regarding the online help and the messages displayed on the screen (score=3.5/7). They also state that the information provided with the system is not easy enough to understand (score=4.5/7), corroborating the case studies and interviews that revealed that the location of the line names could be different and that the units of measurement could be misleading to the extent that they do not correspond to the language used in their work: "I'm not just talking for myself but faced with this, no-one's going to really understand much: what is 0.9; 1.1...", "this voltage thing here: speaking in units is not much use. It's not clear to everyone: well, you put *****". The scores obtained on the AttrakDiff highlight differences in perception in terms of pragmatic quality 4 of the system: typically, the GRD technical manager considers this system to be more technical than human, while the technical assistant considers it to be somewhere between the two. The observation is identical with regard to the dimension of 'hedonic quality' 5 : While both operators find VERTPOM-BANK® particularly original, creative and captivating, the GRD technical manager considers the system to be very bold and innovative, while the technical assistant has a more nuanced opinion. Regarding the overall attractiveness of the system, the operators assess it somewhat positively (score = +1.5 on a scale between -3 and +3). However, some improvements can be made so that the tool better meets their needs and expectations, whether in terms of usefulness and usability, which Hassenzahl (2010) associates with the "do-goals" (e.g., "rebalance the network"), or in terms of the "Self", which Hassenzahl (2010) associates with the "be-goals" corresponding to the reason for using the system (e.g., "to be more efficient", "to be more competent").

Objective
The consumer-centered assessment of the private web portal aims to guarantee a certain compatibility between the web portal and the consumers' EDM activity, and thus to lead to eco-responsible behavior. More precisely, it is about revealing the axes of improvement related to the usability, the functionalities, and the uses of this web portal in connection with the EDM activity.

Methods
To achieve this objective, three methods were deployed: ergonomic inspections, user tests and the persona method. Ergonomic inspections and user tests were carried out at two distinct moments of the design process: before the design and during the design process. Ergonomic inspections -mobilizing Bastien and Scapin's (1993) criteria, Nogier's (2008) recommendations, and Ratier's (2005) guide of ergonomic recommendations for websites and applications -were carried out to identify strengths and weaknesses in order to suggest recommendations (Lallemand & Gronier, 2018). User tests were also conducted to understand how users engage with the web portal via their actions and attitudes towards it. They represent a real opportunity to assess not only usefulness and usability but also UX through validated questionnaires, such as the DEEP or Design-oriented Evaluation of Perceived usability (Yang, Linder, & Bolchini, 2012;Lallemand & Gronier, 2018) and the AttrakDiff (Hassenzahl, 2003;Lallemand et al., 2015). Finally, the persona method was used to obtain a specific and concrete representation of the target users (Pruitt and Adlin, 2006) by attributing to these personas a history, objectives, and a personality, and thus focusing on their needs and expectations (Cooper, 1999). "A persona is a kind of proto-concept, or a configuration of social, affective and cognitive information of an individualized nature: social, because they always situate users in a professional or domestic context, which gives meaning to their action; affective, because they humanize users or customers by giving them a face, values, a name, a singular history; cognitive, because they allow the designers to generate ideas, think about functions, and, finally, deduce specifications about products that would be useful to personas" (Bornet & Brangier, 2013, p. 126 ).
The consumer-centered assessment of the private web portal took place in three stages. Firstly, before the design of the web portal, an ergonomic inspection of two web portals for monitoring consumption (other than the VERTPOM® portal, which had not yet been designed) was carried out by an ergonomist. This allowed the drafting of a first set of specifications addressed to computer scientists for the design of the v1 prototype. This v1 prototype was then assessed in the context of user tests with 6 participants, the results of which led to a second version of the specifications intended for the CIAC-IT partner in charge of the design of the web portal, which constituted the v2 prototype.
In a second step, 8 personas (eg, Figure 5) were constructed based on the interviews conducted by the CURAPP partner. According to Cooper (1999), the primary persona corresponds to the profile of "the one for whom the system will be designed, the one whose needs and goals will be fully met" (Lallemand & Gronier, 2018, p. 309). The secondary persona is a different profile from the first, but has some "common goals and needs with the primary persona" (Ibid.). Finally, the negative persona that is not targeted in the design, but involves "keeping in mind for whom the system should not be designed" (Ibid.), and could also allow for an imagined approach, functionality, etc... in the project. These personas were described using the following headings: a first and last name to humanize the persona, a nickname to quickly access a first representation, their hierarchical location among the personas (e.g., primary persona), an excerpt of the persona's history, and a description of the persona's role in the project: primary persona), an extract of the verbatims from the sociological interviews that best characterize them, personal information (age, profession, family situation), the most salient traits of their personality, a biography describing their history, the context of their use of the system, and their way of thinking, their needs and objectives, the elements that motivate them and those that frustrate them with regard to the use of the system, the salient behaviors of the persona -with regard to the EDM activity -represented on axes (Lallemand and Gronier, 2018).

Figure 5. Example of a persona
In the third step, the v2 prototype was assessed by means of an ergonomic inspection performed by an ergonomist and user tests. These tests involved a sample of eleven participants, including two Gazelec consumers who responded positively to our request and nine newcomers who were recruited on the basis of the profiles highlighted by the personas. The procedure used was the same as the one used for the user tests of the v1 prototype, to which a semi-directive interview was added to allow the users to express themselves more freely in a dynamic discussion with the interwiewer, thus providing qualitative and more detailed data about their EDM activity. The user tests and interviews were recorded and transcribed in order to be analyzed in detail.

Results: Assessment of the prototype v1 of the web portal
The user tests of the v1 prototype (Figure 7) show DEEP scores suggesting an overall positive assessment of the private web portal. Moreover, according to Lallemand & Gronier (2018), the threshold value at which it is necessary to correct a usability problem is 3. In the case of the v1 prototype, the data collected shows that no average value is lower than 3 ( Figure 6).

Figure 6. DEEP Scores
However, these confrontations of the users with the prototype v1 of the web portal reveal that a certain number of difficulties encountered by the latter are likely to be corrected. In total, 6 recommendations were submitted. For example, 4 out of 6 users were unable to find the sub-heading "Energy management", suggesting a problem of visibility of this functionality, which is at the bottom of the long drop-down menu of the heading that users do not always read to the end. This led to a recommendation to put this sub-heading in the navigation bar as a section.

Results: Assessment of the v2 Prototype of the Web Portal
The v2 prototype was first subjected to an assessment by ergonomic inspection. The login page consists of two distinct areas ( Figure 8): (1) an area intended for login and (2) an area intended to inform the user about the different functionalities of the web portal. This page was the subject of eleven recommendations. These recommendations concerned guidance (e.g., integrating a search bar in the navigation bar), accessibility (e.g., setting up tooltips that can be deactivated by the most experienced users in the "settings" section), error management (e.g., being more precise in the connection error message), the workload (e.g., for the consumption graph, proposing a drop-down menu for the choice of the temporality so as not to have to go through all the possible choices) and the meaning of the codes (e.g., adopting the tree structure co-constructed with the users during the first user test session). The user tests of the v2 prototype ( Figure 8) reveal that the participants' assessment of the latter seems relatively nuanced. While the interface, which was a priori uncluttered (e.g., low number of steps to reach a goal, presence of rapid commands), contributed to a positive appraisal of the workload following their manipulations, the participants reported a certain frustration with the lack of explanations of the various functions. Several ideas were put forward, such as the addition of an "Understand consumption data" button and a Frequently Asked Questions page. In terms of usability, consumers suggested adding certain elements that would make navigation more fluid and pleasant (e.g., "You see too much white and green:

I don't know what you call it... It's quite complicated because for me, it's either too white and not green enough, but at the same time the green is a bit harsh on the eyes: it's bright fluorescent green").
At the end of the simulations, it was found that the web portal was assessed as relevant for monitoring consumption and implementing procedures. However, there are gaps in the support of EDM activities (e.g., "Well, in terms of energy consumption control, I don't really see how it can help me. Because, OK, it gives me the consumption etc....

I don't see how I can save money with the information I have, I don't understand".)
The assessment of the user experience using the AttrakDiff scale (Hassenzahl, 2003) indicates that the participants appear to be relatively neutral, with scores ranging from -1 to 1 for each of the subscales (Figures 9.a and 9.b).
The participants felt that the web portal allowed them to achieve certain objectives, such as tracking consumption or administrative procedures, but had some weaknesses with regard to the EDM (PQ = 0.039). The Hedonic Quality Identification (HQI = -0.429) suggests that the web portal is perceived as too "simple" in terms of diversity of tasks proposed, while the Hedonic Quality Stimulation (HQS = 0.662) suggests that the web portal is seen as jsd.ccsenet.org Vol. 14, No. 4;2021 69 too ordinary or even boring. The assessment of the attractiveness of the web portal (ATT = 0.052) suggests that improving the pragmatic and hedonic qualities of the web portal would positively influence its attractiveness. With regard to usability, some points were raised by the participants. Some participants said that they were disturbed by the general layout of the web portal, which seemed too different from the standard websites they are used to (e.g., the navigation bar occupies half of the header; the footer is devoid of any information such as contact information, links to the legal notice and privacy policy). This is consistent with the results of the DEEP usability scale with scores of 2.94/5 for the "Structure" subscale and 2.76/5 for the "Consistency" subscale. The DEEP shows that the first version of the web portal is perceived more positively than the second version ( Figure 10).  Figure 10. DEEP scores for versions 1 (blue) and 2 (green)

Objective
The consumer-centered assessment of the IBox (Figure 11) aims not only to confirm their needs and expectations with respect to the VERTPOM® private portal, but also to analyze the EDM practices -in a natural context (i.e., in situ) -of the consumers. Figure 11. IBox

Methods
Logbook: The logbook method was chosen for this study (Figure 12). This method consists of completing a logbook over several days, allowing for "a repeated assessment of the UX with the same group of users over an extended period of time" (Lallemand & Gronier, 2018, p.626). It has two major advantages: it allows access to data in the context of natural use (and not in the laboratory) on the usefulness, usability, UX of technologies and, consequently, identification of the factors underlying their acceptance on the one hand, while on the other hand it is relevant for collecting behavioral data such as EDM practices since the consumers are free to report their experience without being directly observed by the ergonomist. This alleviates the Hawthorne effect (Mayo, 1933), according to which people who are directly observed are likely to modify their behaviour.
More generally, the logbook is optimal for tracking the temporal dynamics of UX (Karapanos et al., 2009) and for jsd.ccsenet.org Vol. 14, No. 4; identifying existing links between user mood, context of use, utility, usability, and UX. The UX is understood here as a process ranging from anticipation to emotional attachment (Karapanos et al., 2009). Anticipation leads to expectations about the product that will play a role in the experience and the judgment of the quality of the product. The first few times the product is used, the authors call the "orientation" phase: its development depends on learnability (or ease of learning) and stimulation through feelings: notably of excitement and frustration. With increasing use, the growing familiarity with the product leads to the orientation phase during which long-term usability and usefulness play an important role because they induce functional dependence (i.e., the user feels the need to use it in every activity that it can be used for). This leads to the third phase: identification. At this stage, the product is accepted in daily life and plays a role in the construction of social identity, in the sense that owning and using the product -in our case, the IBox -makes the user belong to a group. This belonging allows one to situate oneself in relation to others and encourages interactions between people in the same group (exchanges, solidarity, etc.) in order to reach a common objective that could be defined here as having optimal EDM practices. Figure 12. Excerpt from the logbook The logbook made it possible to collect socio-demographic data to characterize the sample that took part in the study. It also made it possible to collect factual data such as the date, the type of activity carried out, and the modalities of the activity according to a sentence completion type format fostering a feeling of involvement among consumers (e.g., "The data from my IBox was used to..."), as well as subjective data such as perceived usability measured via the System Usability Scale (Brooke, 1996 ;Lallemand and Gronier, 2018) or emotions collected via the Affective Slider or AS scale (Betella and Verschure, 2016; Figure 13) -which is an "updated" and pictorial version of the Self Assessment Manikin or SAM (Bradley and Lang, 1994). A strong significant correlation between SAM and AS scores was demonstrated by Bettella and Verschure (2016). The significantly increasing popularity of emojis in commication systems (Guibon et al., 2018) suggests that the use of emojis in emotion measurement scales (eg, the AS) seems more relevant than using the SAM. Finally, there were also note pages available for consumers to provide additional feedback. Figure 13. Affective Slider (Betella and Verschure, 2016) Event and interval testing protocol: The mixed-mode protocol -by event and by interval -seemed to be the most adapted to obtain data concerning the UX in the context of the user's interaction with the IBox and those concerning the consumers' EDM practices. The event-based handover protocol consists of the consumers reporting their experience in the logbook each time they perform an activity related to EDM (e.g., consulting their consumption, performing a search). This method allows tracking of the temporal dynamics of the activity as the IBox is used.

Journal of Sustainable Development
In a second step, an interval-based logbook completion was proposed. Consumers had to complete the logbook every Sunday, from the first use of their IBox until the end of the study, to determine the progression of their practices.
Consumer commitment while completing the logbook: Despite the advantages of the logbook method, its implementation requires a certain commitment from participants. In this regard, out of the 40 consumers contacted, only two volunteered to take part in the study. Indeed, integrating a logbook into the daily life of individuals is not a trivial matter because it puts a constraint on them, in addition to the risk of modifying their activities by focusing on them (Lallemand & Gronier, 2018). It is, therefore, essential to implement strategies to keep as many participants as possible until the end of the planned period sufficient to obtain a representative number of exploitable data (in our case, three weeks).
To address these limitations, each participant was contacted individually before the logbooks were handed over. During this exchange, the objective of this approach, their role in the study and our expectations of them, as well as the presentation of the use and content of the logbook were discussed. In addition, to guarantee that the consumers would find it easy to complete and understand, the logbook was pre-tested with five people.

Results: Characteristics of consumer volunteers
The two consumer volunteers were men with an average age of 69 years. One is retired and the other works in logistics. While they both agree that they prefer using a computer -with which they are generally comfortable -as a consumption monitoring tool, they would also like to be able to use the IBox.

Results: IBox Experience
The IBox was used to calculate the daily consumption of one of the consumers and to see the distribution of consumption according to peak and off-peak hours for the other one. Consumers had only two types of physical interaction with the IBox: pressing one or more buttons and looking at the screen. They said they appreciated the presence of the screen, which allowed them to read the display directly. In this respect, one of the consumers was able to determine, thanks to his IBox, that his weekly consumption was regular, and that there was a seasonal variation in consumption. However, he disliked the lack of precision and real-time display, as well as not being able to determine his consumption by type of appliance: "If the dial shows 10, I don't know if I'm at 10.1 or 10.9:  = 0,8. If the dial shows 11, the  could either be 0.2 (10.9 to 11.1) or 1: between 10.9 and 11.9? It is obvious that the display is not very accurate when it comes to calculating consumption, and it is impossible to know what the various household appliances, lighting or recreational appliances consume". To make up for this lack of functionalities on the IBox, one of them asked for the addition of another box to display consumption in real time.
Beyond these shortcomings in terms of the IBox's usefulness, they also pointed out usability problems. Indeed, they would like it to be easier to use or, failing that, that an explanatory user guide be provided, to better understand how to use the buttons. In the same vein, consumers also mentioned the poor legibility of the information on the IBox screen, due to the lack of backlighting and the use of English.
to allow a display according to the user's preference with regard to the desired degree of measurement precision; to implement backlighting of the IBox screen; to allow a real-time display of consumption; and to propose a system in French.
Beside these specific elements of the IBox properties, one of them expressed his dissatisfaction with the location of the box making its use uncomfortable (ie, "my meter is badly placed, I can't see anything, not even the screen"), while the other expressed the need for an aesthetic improvement of the IBox because it is located in full view of everyone in the house (i.e., "the IBox design needs improving, as it's not in a cabinet [...] a casing would be a good idea to cover up its appearance"). This last point reminds us that today, these technological tools are also elements of home decoration. It is therefore important that they have a "pleasant and attractive aesthetic for the user and respect the standards of usability" (Lallemand and Gronier, 2018, p.361). Kujala et al. (2011) highlighted product aesthetics as one of the factors for rapidly improving product attractiveness.

Discussion
The objective of our paper was to propose a user-centered assessment approach to design useful and usable Smart Grids with a good UX in order to promote their acceptance, and to illustrate the use of such an approach in the VERTPOM® project. In this project, this approach was applied to two complex technologies and mobilized two types of users: the VERTPOM-BANK® software intended for DSO operators and the "private web portal / IBox" system intended for consumers. Three methods were used: ergonomic inspection, user test and logbook (Table 1). This feedback shows the interest and necessity of involving users in the design of Smart Grid technologies. Indeed, at the end of the user-centered assessment process presented, 36 recommendations were proposed for the VERTPOM-BANK® software, 35 recommendations were suggested for the private web portal and 8 solutions were mentioned for the IBox.
It also demonstrates the relevance of methodological crossover in the design of innovative technologies (Loup-Escande & Loup, 2020). Moreover, it seems judicious to implement an assessment method that does not involve end-users (eg, ergonomic inspection) in order to identify the major usability problems of a version 1 of a prototype in order to propose recommendations to be implemented in a version 2 so as to limit the risk of rejection of the technology by the real end-users (Laumer & Eckhardt, 2012), as was done for the consumer-centered assessment of the web portal. The assessments of the VERTPOM-BANK® software and the private web portal also show that user tests and usage situations are necessary to access objective and subjective data on the relevance of functionalities, usability difficulties associated with usability issues, and UX-related dimensions (Loup-Escande & Burkhardt, 2019). More precisely, while the ergonomic inspection of the VERTPOM-BANK® software made it possible to provide elements of correction to optimize the interface initially, the user tests proved to be conducive to discussions with the operators, thus making it possible to identify their expectations and to better situate the context in which the software will ultimately be implemented.
Despite these clear contributions, two limitations directly related to the role and the place dedicated to usercentered assessment should be emphasized and lead us to suggest research perspectives for future work. The first limitation relates to methodological choices. While this feedback confirms the need to combine expert assessment methods (e.g., ergonomic inspection) and assessment methods that directly involve users (e.g., user tests, logbooks), all three assessment methods (an ergonomic inspection, a user test, and an in-situ analysis using logbooks) should have been systematically deployed for all three technologies.
The second limitation relates to the timing of the user-centered assessment. In the VERTPOM® project, the assessment of the VERTPOM-BANK® software was carried out during IT development. Unfortunately, it was not possible to carry out a study before the IT development for two reasons: on the one hand, no comparable software on which we could have carried out the preliminary assessments exists, and on the other hand, it was not possible to carry out an analysis of the activity of the DSO operators, which would have made it possible to identify mismatches between the needs of the operators and their implementation in the software (e.g., the configuration of the network mapping, which differs from the existing one). In the same vein, it was not possible to assess the final software in situation since it was not deployed at the DSO partner's site within the project timeframe. The assessment of the private web portal was conducted upstream and during the IT development but could not be implemented downstream of the project since at the end of the project, it had still not been deployed to consumers. Consumer-centered assessments after the design of the web portal would have been particularly interesting to investigate the gamification potentially conducive to EDM activity, in that it can foster user engagement by making usage more stimulating. According to Lallemand & Gronier (2018), it is a "powerful motivator" and "can help individuals adopt healthier, greener behaviors, or engage in humanitarian causes" (ibid., p. 375). Being based on self-determination theory (Deci and Ryan, 1985) and flow theory (Csíkszentmihályi, 1996), gamification includes "powerful factors to optimize a user's well-being and experience of the interaction" (ibid., p. 375) such as "fulfilling basic needs (competence, anatomy, relational) or achieving a state of flow" (p.375, ibid.). As far as the IBox is concerned, even if the assessment carried out in the consumers' homes shows that it corresponds to their needs in terms of communication of energy consumption, the fact that we were not able to intervene earlier (i.e., neither upstream nor during the development) in this design task, which was rather oriented towards solving technological problems, did not allow us to design a real integrated system (which would have incorporated the functions of the IBox and of the web portal) in order to support EDM activity. Yet, it is important to mobilize users as early as possible in the design of technologies, from the upstream phases (i.e., before IT development) to their in-situ implementation (Loup-Escande, Burkhardt, Christmann & Richir, 2014). On this last point, a future research perspective concerns the longitudinal analysis which would have a double objective: to accompany the change induced by the introduction of these new technologies in the business activities of DSO operators and in consumers' EDM activities on the one hand, and on the other, to help develop the technologies according to real user practices (Rabardel & Beguin, 2005).

Conclusion
Based on an empirical research within the VERTPOM® project, this paper shows that the design of Smart Grids implies considering the human-related issues. Indeed, it proposes the integration of DSO operators and consumers within a user-centered evaluation approach in order to design Smart Grids that are sufficiently acceptable to users to enable Positive Energy Territories. The user-centered evaluation approach (with an ergonomic inspection, a user test, and/or an in-situ analysis using logbooks) was applied to the VERTPOM-BANK® supervision tool intended for DSO operators, the private website and the IBox smart meter intended for consumers, from the upstream design phase to the implementation of the technologies in real-life situations. In future work, a longitudinal analysis of real user practices induced by these emerging technologies in the activities of DSO operators and in consumers' EDM activities will be conducted to improve the technologies.