The Quantification and Reporting of Negawatt-Hours with Flexible Energy Conservation Measure Verification Software (ECM-Tool)

In order to promote digital innovations in the field of energy use and monitoring in all end customer sectors, the Federal Ministry for Economic Affairs and Energy (BMWi) has launched the “Pilotprogramm Einsparzähler” in 2016. The program promotes the development of digital platforms following the “Efficiency First” principle, focusing not on individual projects but on the establishment of a business model. smartB successfully applied for subsidies for the development of a software tool, the architecture of which is the con-tent of this open source paper. The tool applies a multivariate regression-model to model a given system’s energy consumption (significant energy uses or SEUs), adjusted to relevant external factors (e.g. weather) and given output levels or product properties. Thereby comparing energy consumption before and after an energy conservation measure (ECM), the tool allows for a quantification and verification of achieved energy savings as laid out in international standards for energy management (ISO, 2014). Achieved energy savings induced by an ECM and energy efficiency improvements cannot be measured directly. We use the term “negawatt-hour”, defined as a unit of energy saved as a direct result of energy conservation measures. International norms provide accepted standards to derive quantified savings in nega-watt-hours from a qualified comparison between consumption before and after an ECM, as presented at the beginning of the paper.


Introduction
In the year of 2016, the Federal Ministry for Economic Affairs and Energy

Norms and Context: IPMVP and ISO 50006 to Achieve Efficiency First
The Federal Ministry of Economic Affairs and Energy (BMWi) set forth a strategy for energy efficiency policy in Germany in the 2016 "Green Paper on Energy Efficiency" [3], which has since received critical feedback in a stakeholder consultation process. The conclusions of this process will lead to a white paper on energy efficiency, soon to be published. The first thesis emphasizes the central goal to save energy: "Efficiency first leads to a cost-optimal energy transition and reinforces renewable energy's effect on decarbonisation. A unit of energy saved need not be produced, stored or transmitted over the grid". (BMWi, 2016) In the same year and in order to promote digital innovations in the field of energy efficiency and monitoring for all end customer sectors, BMWi launched a funding support scheme called "Pilotprogramm Einsparzähler (i.e. pilot program energy savings meter)". The program promotes the development of digital platforms following the Efficiency First principle, focusing not on individual projects but on the establishment of a business model. The funding authority BMWi established the condition sine qua non for any project in the Pilotprogramm Einsparzähler to offer a product with hard and software that presents the end customer with a transparent and reproducible quantification of energy efficiency. Comparing energy consumption before and after an energy conservation measure (ECM) allows for a quantification and verification of achieved energy savings as laid out in international standards for energy management such as the International Performance Measurement and Verification Protocol (IPMVP) [1] as well as ISO 50001:2018 [4], ISO 50006:2014 [2] and ISO 50015:2014 [5], respectively. Measurement and verification (M&V) is a prerequisite for all performance-based energy-efficiency projects to assess and audit the quantitative outcomes of energy conservation measures (ECMs). The IPMVP provides a rigorous and yet flexible framework for evaluating the performance of an ECM and is the most prevalent M&V methodology employed worldwide. The related alter-native standard increasingly applied in Germany and therefore the focus of this paper is the ISO 50006:2014. This norm standardizes process and methodology in energy management to provide evidence of improved energy-related performance. The consumption of essential significant energy uses has to be adjusted by factors such as weather or product properties.
Both the core concepts and general methodologies in ISO and IPMVP are robust and show little distinction. The most significant drawback in these norms is the widely published [6] lack of guidance on the calculation process. This issue is more present when performing M&V in industrial facilities, where the quantity of factors impacting on energy performance complicates the modelling process. The ISO 50006 might be considered to complement IPMVP although there is not an official link between them. ISO 50015 actually compliments ISO 50001 and like IPMVP, ISO 50015 sets out to establish a common set of principles and guidelines to be used for measurement and verification of organisational energy performance.
ISO 50006 is the pertinent norm for smartB's ECM-tool, since it guides organizations on measuring energy performance using energy baselines (EnB) and energy performance indicators (EnPI), which in turn is fundamental in managing energy performance with regard to ISO 50001 and ISO 50015. An En-PI defines a quantitative value or measure of energy performance for a given system's significant energy use (SEU), which can be derived as (a) an absolute value, e.g. energy use in kWh over a certain time span; (b) a ratio of values, e.g. energy use in kWh per unit of output; (c) a statistical model, e.g. energy use as a regression function of weather conditions and output; or (d) an engineering model, e.g. energy use modelled with physical properties in any functional form. (a) and (b) provide the clear advantage of simplicity, which in turn might fail to adequately indicate multiple dependencies between energy use and relevant variables. Therefore only (c) and (d) allow for the normalization of an SEU to routinely modify energy data in order to account for changes in two or more relevant variables to compare energy performance under equivalent circumstances.
The EnB is the quantitative reference providing a basis for comparison of energy performance in the baseline period before an ECM versus the reporting period after an ECM. In case of (c) a statistical model, the EnB consists of a set of parameters used to forecast energy consumption in the reporting period, based on the functional relationship in the baseline period between energy use and relevant variables, had there not been an ECM. Apart from normalizing with respect to two or more relevant variables, this forecasting method gives a hypothetical energy consumption curve in the reporting period. By controlling for the influence of relevant variables and static factors on a system's energy performance, the statistical model facilitates the interpretation of a change in energy consumption as a causal effect of the ECM. The amount of energy saved can be derived from the area between two curves (i.e. ΔE in the following graph): the forecast energy consumption and the actual energy consumption in the reporting period after the ECM. Figure 1 visualizes this approach in two-dimensional

A Glance at the Market for Energy Management Systems
Supporting ISO 50006

German Market Participants
Energy management software is the key to keeping companies lean, efficient, and sustainable. Recent advancements in technology and IT infrastructure make implementing these solutions easier, and more attractive than ever 1  Furthermore, the definition of targets makes possible to get clarity in every kind of deviation that could happen and making possible to react and correct the problem that could have lead to this.

International Market Participants
Outside of Germany these companies provide energy management solutions:  During the creation of an energy model the user can choose not only linear forms but polinomial, exponential and logarithmic, making the tool ideal for complex cases where not always a linerat solution can be found. Nevertheless, uploading data to the system is not easy and is needed to merge data from one database to another. Even if many graphs options are included to visualize the data, they are in a very old school way, since the focus of the software is more oriented in its functionality.

Unique Selling Proposition (USP)
As exposed in the previous chapter, there is a variety of software solutions available on the market that allow the user to calculate energy savings based either on the IPMVP or the relevant ISO norms. Nevertheless, every one of them has its own limitations, which translates to a lack of flexibility and efficiency for the user.
Software offered by other market participants uses diverse M&V methods to calculate savings. In many cases, these methods are implementations of industry-standard approaches, such as those described in the IPMVP [1] or those usually used for evaluating efficiency programs [7]. Tools may differ in 1) whether they describe what they calculate as gross or net savings, 2) in the regression approach to calculate the energy model, 3) the method to determine savings, or 4) in their ability to operate on whole buildings as well as submetering data. In addition, some tools are programmed to report accuracy metrics such as baseline model goodness-of-fit, or estimations of savings uncertainty [8].
Since the quantification of energy savings often is not the focus of existing energy management software, respective modules are often complicated to use, not user friendly and intuitive. In order to set up data handling and modelling to derive savings, the user needs a high level of expertise, read a manual and attend instruction classes or online seminars, which generate extra costs and barriers to entry. Usability benefits from seamless data integration via an application programming interface (API), which allows scaling the solution to many projects, without manual data import and export from other sources. Tools should also allow the user to set the baseline period, the date of the ECM and reporting period manually.
Data resolution can be an issue, since many solutions on the market do not support high frequency data. Two to fifteen minute intervals between observations are a common default resolution. Nevertheless, in order to find better correlations and therefore more accurate energy models some cases mandate a With this set of USPs smartB's ECM-tool solves the problem to calculate energy savings, encountered by many energy consultants. Office software such as Microsoft Excel, the favorite tool of many German engineers, provides a similar functionality. However, the ECM-tool potentially digests much more data significantly faster and visualizes data automatically. Furthermore, using the ECM-tool is more reliable and reproducible, since calculating point estimates and prediction intervals for accumulated energy savings can be a cumbersome manual effort and therefore prone to errors. Figure 2 demonstrates a mock up of the ECM-tool proposed above.

The Scope of the ECM-Tool
We separate three phases in energy management, of which only the second phase is the scope of the ECM-tool: • Definition of energy system and energy performance indicators including data gathering as well as energy efficiency measure planning and implementation.
• Quantification of energy savings using energy baselines and a statistical model as well as the interpretation and reporting of energy efficiency improvements. Open Journal of Energy Efficiency • Maintaining energy performance indicators and continual improvements of energy efficiency as well as preparation of management reviews and decision models.

Preparation by the User
System boundaries and model definition: Phase 1 implies preparation by the user, before the ECM-tool yields a quantification of energy performance of a given system. The selection of time series to include in the data set is driven by theoretical considerations based on the type and boundary of the energy system which received an energy conservation measure. The approach to use regression models as EnPIs applies best to subsystems with clear boundaries between SEUs, such as lighting, an (electrical) heating register or a ventilation system. The regression model should include all relevant variables for all subsystems covered by the system boundaries. If the system boundaries include several SEUs, a relevant variable might not impact each SEU equally, which introduces noise and inaccuracy. Therefore, the fewer SEUs the system covers, the more accurate the regression model predicts energy savings. An energy system with suitable boundaries shows high correlation between energy consumption and relevant variables with high predictive power as measured in high R 2 and narrow prediction intervals 4 .
Date of ECM implementation: The core feature of the ECM-tool is to quan- 4 For instance, consider system boundaries including lighting and a heating register, both on the same electricity meter. Since we expect a relevant influence of outside temperature on the electricity uptake of the heating register, the model should include average temperature as an explanatory variable in the model. However, we expect no predictive power of outside temperature on the electricity consumption of the lighting system. Therefore, separate models for each SEU (e.g. heating, lighting, etc.) deliver more accurate quantifications of energy savings, which is only possible if each subsystem is measured with separate electricity meters. Open Journal of Energy Efficiency tify savings as a direct consequence of an energy conservation measure implemented at a certain date. However, the ECM-tool can as well be applied to simply compare year over year energy consumption by arbitrarily setting the time frames for baseline and reporting periods accordingly.
EnPI definition: Whenever the ECM-tool is used to quantify savings, the En-PI is defined as the set of coefficients from the regression model over the baseline period. In case of electricity, the SEU can be denoted in units work (e.g. kWh) or units of power (e.g. kW). Since the ECM-tool shows quantified savings in the same unit as the dependent variable, kWh is preferred, which allows a direct interpretation of the output without manual follow-up calculations.
Suitable baseline period: The user should provide sufficient time frames for the baseline period and the reporting period, to derive reliable parameters. In general, a model predicts reliably, if relevant variables exhibit the same range of values during the baseline period as well as during the reporting period. In cases of a production process or a lighting system, data over some weeks of observations in the baseline period often fulfill this criterion. However, an accurate model of the correlations between an SEU and outside temperature in a thermal process (e.g. heating) should include warm and cold seasons to fully capture the system's behavior over one full year. For instance, if the baseline period covers temperatures between 10˚C -25˚C, a prediction of expected energy consumption for temperatures below 0˚C in the reporting period hinges on the assumption that the correlation between temperature and consumption extrapolates linearly. Therefore, whenever the energy system provides thermal energy the influence of local weather conditions (in terms of average temperature) must be tested and the baseline period adjusted accordingly.

The Architecture of the ECM-Tool
In phase 1, the user prepares a .csv data-file, which meets the criteria defined above. In phase 2, the user runs a statistical analysis on this data to calculate savings and interacts with the ECM-tool on two main user interfaces: the landing page and the model page.

The Landing Page
The landing page shows the signup and an email verification feature using a web-token. The user opens an existing project or names and creates a new project by uploading one or more .csv-files containing time-series data. The tool shows an error, if it cannot read the .csv-file or if it is empty. With a click the user opens the project, which triggers the tool to check data for consistency and discard incomplete or non-numeric columns in order to create clean data to be used on the following screen for regression modelling.

The Model Page
The model page shows four elements: data selection, timeframes, data visualization and model output.

Technical Implementation of the ECM-Tool
Data handling and parsing are implemented based on the programming language "Python 3.7" 5 with the popular "Pandas 0.24" 6 library for data manipula-  Ridge regression: The ECM-tool applies an estimator called the ridge regression, as implemented in the python library "scikit-learn" 8 . Compared to ordinary least squares, regularization with ridge regression has some desirable properties.
Ridge regression slightly shrinks the coefficients, which on the one hand introduces bias in the estimation, but on the other hand reduces model complexity and multicollinearity. At the same time, using a method called the cross-validation

Integration of Results in the Context of the Organization
In phase 3 the user integrates insights from the ECM-tool in a continual process to improve energy efficiency organization-wide, as outlined in ISO 50001:2018.
The ECM-tool supports the monitoring and verification of changes in energy performance of subsystems to judge the effectiveness of an ECM. The output easily translates into associated economic gain, by comparing investment costs and the monetary value of quantified savings under equivalent circumstances.

The Energy Performance Report
We present three anonymised cases to quantify achieved energy savings by using the ECM-tool. Each case mandates specific preparation of the data set and poses unique properties. All consumption data used unaltered for this paper was generated by real electricity meters in the field and the presentation in this paper in its current form is legitimised by the owner of the data. Each of the three cases presents particular features of the ECM-tool as well as a discussion of the quality and reliability of savings estimates.

ECM-Tool Application-The Performance Report
Elements from the ECM-tool can be seen in Table 1.

Interpretation of the Model
As expected, the model based on dummies shows high variance in the prediction but sizable relative savings. We accept the model to reliably quantify energy savings.
Baseline period: Modelling the 24/7 car park is a very special case, since in order to quantify savings, the energy system is sufficiently specified without relevant variables or influencing factors. In regression modelling, the constant is  Table 2 shows the most important statistics for the evaluation of the models.

Preparation
System boundaries: This case looks at the ventilation system of an office building in the north of Germany with daily consumption of electricity in kWh over the time frame 11th September 2017 until 24th July 2019. The system boundaries, as seen in Figure 4, include the ventilation system that applies electrical power for the operation of the fans, butterfly valves, valve drives and water pumps. All power consumers are high efficiency devices. Outside temperatures do not influence the power consumption of the system. The building is connected to a local heating and cooling network with generation at a central location in another building on site. The electricity uptake of the water pumps is negligible compared to other applications on the same meter and can therefore not be detected separately. The ventilation system runs various predefined operating modes depending on the day of the week. ECM: The system received a total of two energy conservation measures. The first one, a decrement in the operational time of the system was implemented on 2nd November 2017 remaining as follows:  Saturday-Sunday off.
Public holidays off.
The second ECM was a filter replacement, which had no significant effect on the energy consumption of the systems. On the one hand, this is due to the regular change intervals of the filters, on the other hand, the efficiency improvement might be too small to be detected with the available data and the model presented here. One whole year of baseline data was taken for the calculation of the energy model. out of 8 potential dummy variables were taken into consideration.

ECM-Tool Application-The Performance Report
Elements from the ECM-tool can be seen below in Table 3.

Interpretation of the Model
The presented model shows very good fit and little variance in the prediction.
Quantified savings are reliable and precise.
Baseline period: Even without weather influences on the system, the baseline period of one year of data between 1st November 2017 and 1st November 2018 yields a robust model with little variance that covers all holidays and special cases (i.e. interruption of the system due to maintenance).

Preparation
System boundaries: Case C is based on a cooling system with suboptimal data availability. The SEU is measured in kWh electrical work including weather data as a relevant variable. The output of the cooling system in thermal energy is not available. Instead, we use electrical power consumption of the connected secondary system as a proxy, which takes the cooling energy as an input. Furthermore, there is little information about the context of the cooling system and the secondary system apart from the location in an industrial production setting.
The reasons to include this example despite these serious drawbacks are twofold: On the one hand, this case proves that a statistical model with favorable properties can fail to convince the user of the reliability of potential savings without de- as well as the energy consumption of the secondary system. The cutoff temperature for CDD (17˚C) was selected to provide the best correlation between CDD and energy consumption using a third party statistic tool.
The model: Y = a + b1 × (kWh of secondary system) + b2 × ˚C. Open Journal of Energy Efficiency with R 2 increasing marginally in the fourth digit. However, the complexity of the interpretation of coefficients from regressions model including higher order terms (e.g. cubed CDD) rises exponentially. Therefore, we select the model with only two variables: outside temperature and energy consumption of the secondary system as a proxy for cooling demand.

ECM-Tool Application-The Performance Report
Elements from the ECM-tool can be seen below in Table 4.

Interpretation of the Model
Baseline period and energy savings: The observation period covers one whole year, which ensures full coverage of seasonal effects and potential variance of outside temperature. For the lack of an ECM there is no reporting period and no savings in this case.
Marginal Effects: The coefficient on X1 implies that energy consumption of the cooling system increases by 0.086 kWh for every increase of 1 kWh in the secondary system. Since X2 is denoted in ˚C × 1000 (e.g. average daily outside F. Milojkovic et al. kWh per day for every one degree increase in average daily outside temperature. Quality of the model: From a statistical point of view, the data and model fulfill all requirements to serve as a reliable baseline to quantify savings. A R 2 = 0.90 implies that a high amount of the variation in energy consumption can be explained by the variation in two independent variables. Furthermore, CV (RMSE) = 0.14 indicates a good model fit in terms of the relative sizes of the squared residuals and outcome values. However, the data shows very sizable consumption (with daily average of > 2 million kWh in the cooling system) and rather little information about context and technical details. Without further information (e.g. a detailed schematic diagram), we do not recommend to rely on savings calculated based on this data. Furthermore, the user would need to check with the system operator, whether static factors remain stable over time. Changes in system setup would have to be identified, such as the area to be cooled or the characteristics of the secondary system. Foremost, however, including time series of the output of the cooling system in kWh thermal energy from a metering system would replace the (necessarily imperfect) proxy variable, kWh electricity uptake of the secondary system, and improve the statistical Open Journal of Energy Efficiency model for better resemblance with the physical properties of the underlying SEU.

Conclusions
The ECM-tool was developed with funding support from the Federal Ministry for Economic Affairs and Energy as part of the Pilotprogramm Einsparzähler.
The tool solves a problem of practitioners in the field of energy efficiency to calculate savings from an energy conservation measure in a convenient, transparent and reproducible way. This paper describes the preparation and process to quantify energy savings based on three cases with real data from electricity meters. ISO 50006 provides the guidelines for the ECM-tool to focus on the usability of a multivariate regression model to compare consumption of an energy system before and after an ECM under equivalent circumstances.
The currently available minimal viable product of the ECM-tool covers modelling and quantification of savings. The assumption here is that the user presents a suitable time series data set for energy consumption and all relevant variables over a sufficient period of time. With little experience in regression modelling, the interpretation of the output from the tool allows the user to judge the statistical characteristics of the model. However, the tool does not cover any meta-information on the energy system under observation or any plausibility checks.
With this paper, we close the chapter of ECM-tool development for the time being for the lack of further funding. However, as this paper argues, there are few solutions on the market to solve the specific problem to monitor and verify energy savings in a complex world. Potentially, the ECM-tool could enter the market as a stand-alone software or as a feature in an energy management software package with wider use.