uncertainty analysis in building energy simulation … · uncertainty analysis in building energy...

UNCERTAINTY ANALYSIS IN BUILDING ENERGY SIMULATION:A PRACTICAL APPROACH

Adrian Chong1, Weili Xu1, and Khee Poh Lam1

1Center for Building Performance and Diagnostics, Carnegie Mellon University, Pittsburgh, USA

ABSTRACTUncertainty analysis in building energy simulation isoften carried out with Monte Carlo Analysis. How-ever, there is currently no standard framework for un-certainty modeling in building energy models. In par-ticular, uncertainty quantification is often based on lit-erature and expert judgment with limited attention onthe use of measured data. The purpose of this paperis to provide a structured framework for modeling un-certainties in building energy models. A method forselecting probability distributions based on measureddata is presented, allowing users to assign probabil-ity distributions that better represents actual data dis-tributions. If measured data is not available, the sub-jective assignment of an appropriate probability dis-tribution would be based on drawings, specifications,literature and past case studies on uncertainty quan-tification. To facilitate its implementation to buildingdesign and retrofit analysis, an application that cou-ples the proposed framework with EnergyPlus is in-troduced. Demonstrated through a case study, the pro-posed framework provides acceptable simulation re-sults in comparison with measured gas and electricityconsumption.

INTRODUCTIONBuilding energy simulation has the potential to guidebuilding design by providing a means to quantify andevaluate the energy efficiency of design alternatives.Calibrated simulation is also recommended for es-tablishing benchmark energy consumption when suchdata is not available or the performance of each energyconservation measure needs to be evaluated individu-ally (EVO, 2012). An accurate model requires users toprovide a detailed description of a building’s geometryand construction, its associated mechanical systemsand the quantification of various internal loads. How-ever, detailed information on building materials, com-ponents and systems specification needed for energysimulation are often not available in reality. As a re-sult, the modeler is often forced to set unknown param-eters to default values and best guesses. Furthermore,the complexity of building systems often make it nec-essary to introduce simplifications into the simulationmodel (De Wit, 2004). These simplifications add tothe uncertainties in the simulation outcome. However,uncertainty analysis is still the exception rather than

the norm.

Although uncommon the use of uncertainty analysisin building performance simulation is not new. This isbecause current tools do not facilitate the inclusion ofuncertainty analysis in the simulation process. To date,several studies have been conducted on the use of un-certainty analysis in building performance simulation(De Wit and Augenbroe, 2002; Struck and Hensen,2007; Hopfe and Hensen, 2011; Eisenhower et al.,2012; Lee et al., 2013). This process often involvesthe quantification of uncertainties in the model’s in-puts, using information from literature, subjective as-sessment or expert judgement. Uncertainties in thesimulation outcome are then quantified using MonteCarlo Analysis. This process is cumbersome and un-suitable for practical applications since it often in-volves a manual changing of inputs or the writing ofspecific programs that needs to be modified each timethe programs are used for a new project. Furthermore,Monte Carlo simulation requires running many sim-ulations and generating many output files that can betime-consuming to process.

Modeling uncertainties based on observed data hasalso received limited attention, with most studies as-suming a Gaussian, uniform or triangle distributionwhich may not be representative of the actual datadistribution. Although Bayesian calibration has beenshown to be effective, it is currently not suitable for usewith building performance simulation programs suchas EnergyPlus and ESP-r due to their large number ofinput parameters (Heo et al., 2012). As a result, theuse of uncertainty analysis has not been widespread inpractice, even though it offers decision-makers greaterassurance in the results generated by simulation tools.

This paper extends current approaches by describing asystematic approach to model uncertainties for practi-cal applications in building performance simulation. Amethod for assigning a parametric probability distribu-tion based on observed data is presented. If observeddata is not available, a knowledge base from past stud-ies and literature is used to guide users on the sub-jective assignment of an appropriate probability distri-bution. To facilitate this process, an application thatcombines uncertainty modeling with the building en-ergy simulation program EnergyPlus is introduced anddemonstrated in this paper.

Proceedings of BS2015: 14th Conference of International Building Performance Simulation Association, Hyderabad, India, Dec. 7-9, 2015.

- 2796 -

Figure 1 Framework to model uncertainty in buildingenergy simulation.

METHODFramework for uncertainty analysisFigure 1 shows a framework for the quantification ofuncertainties in building performance simulation andcan be summarized by the following steps:

1. Determine the parameters that are uncertain in thebuilding energy model.

2. For each uncertain parameter,

(a) First determine if observed data is avail-able. If it is available, fit different paramet-ric probability distributions to the data andselect the one that gives the best fit.

(b) If observed data is not available, determineif a parametric probability distribution canbe assigned using subjective assessment ofavailable information such as drawings andspecifications.

(c) If no relevant information from drawingsand specifications are available, assign aparametric probability distribution based oncurrent standards, literature and case studieson uncertainty analysis in the field of build-ing performance simulation.

(d) Using the parametric probability distributiondetermined by steps 2a to 2c, generate ran-dom samples for the uncertain parameter.For this study and in the developed appli-cation, Latin hypercube sampling (LHS) isused to sample from the assigned probabil-

ity distribution. Because of its efficient strat-ification process, LHS produces results thatare more stable than crude Monte Carlo sam-pling and provides advantages in the analy-sis of complex models with long run-times(Reddy, 2011).

3. Use the random samples generated for each un-certain parameter as inputs to the building energymodel and run the simulations. It should be notedthat if m × n random samples were generated,where n is the number of uncertain parameters andm is the number of samples drawn, there would bem simulation models since each set of n randomsamples corresponds to the uncertain input param-eters in a single simulation model.

4. Process the output files and analyze the results af-ter the simulations are completed.

To facilitate this process, an application that combinesuncertainty modeling with whole building energy sim-ulation program EnergyPlus was created. The use ofthis application will be demonstrated with a case study.

Fitting distributions to dataIf observed data is available, the probability distribu-tion should be constructed based on observations in-stead of using subjective assessment. This can beachieved by fitting different parametric probabilitydistributions to the data and selecting the one thatgives the best fit based on some criterion (Sheppard,2012). Possible criteria include the negative log-likelihood (Equation 1), Bayesian information crite-rion (BIC) (Equation 2), Akaike information criterion(AIC) (Equation 3) and AIC with a correction for finitesample size (AICc) (Equation 4).

− logL(θ|x) = − log

n∏i=1

f(Xi|θ) (1)

BIC = −2 logL(θ|x) + k log(n) (2)

AIC = −2 logL(θ|x) + 2k (3)

AICc = AIC +2k(k + 1)

n− k − 1(4)

θ denotes the parameters of the probability distribu-tion and is determined using maximum likelihood es-timation. X1, ..., Xn denotes the observed data andare assumed to be independent and identically dis-tributed (i.i.d.). n is the number of observations andk is the number of parameters to be estimated. Asan example, if the probability distribution under con-sideration is the normal distribution, then k = 2 andθ = (µ, σ) since the normal distribution is defined bytwo parameters, the mean (µ) and the standard devia-tion (σ). Given X1, ..., Xn, maximum likelihood esti-mation maximizes L(θ|x) over all possible θ.


- 2797 -

For this study, each set of observed data is fitted to 17continuous probability distributions or 3 discrete prob-ability distributions accordingly. The continuous dis-tributions include the Beta, Birnbaum-Saunders, Ex-ponential, Extreme Value, Gamma, Generalized Ex-treme Value, Generalized Pareto, Inverse Gaussian,Logistic, Loglogistic, Lognormal, Nakagami, Normal,Rayleigh, Rician, t Location-Scale and Weilbull dis-tribution. The discrete distributions include the Bi-nomial, Negative Binomial and Poisson distribution.The probability distribution that gives the minimumnegative log-likelihood, BIC, AIC or AICc is then se-lected and used to generate inputs to the building en-ergy model.

Create probability distributionIf observed data is not available, the assignment willbe based on subjective assessment that is derived froma pool of information such as equipment specifica-tions, nameplates, industry standards, expert judge-ment, building surveys, etc. However, if both observeddata and relevant information from this pool of infor-mation are not available, assignment of an appropri-ate probability distribution would be based on litera-ture and past case studies on uncertainty quantifica-tion. This is further illustrated in the case study below.

APPLICATIONCase studyThe building analyzed is an actual ten story officebuilding located in Pennsylvania, U.S.A. The buildinggeometry is created using DesignBuilder V3.2 (Figure2) which is then exported as an EnergyPlus model. En-ergyPlus V8.0 is then used to create all other buildingsystems and schedules.

Figure 2 Geometry of office building inDesignBuilder.

The HVAC system is a dual duct system where bothwarm and cold air are separately ducted and mixed ateach terminal unit to achieve the desired temperature.Cooling is supplied with a water cooled chiller whileheating is supplied through gas boilers. Data collec-tion for this study took place between 16 March 2013and 18 December 2013. The weather file used for thesimulation is the Actual Meteorological Year (AMY)weather data of the nearest weather station for the pe-riod of the data collection (DOE, 2014).

649 parameters were modeled as uncertain in the En-ergyPlus model. This is done by setting the value ofeach parameter to a string that begins with a dollar($) sign (point 1 in Figure 3). Loading the modifiedEnergyPlus input data file (IDF) into the applicationwould then list out all random variables (uncertain pa-rameters) on the left panel (points 2 and 3 in Figure 3).Coupling with EnergyPlus is achieved by reading theIDF and storing each EnergyPlus object as key-valuepairs in computer memory. These key-value pairs canbe modified and exported as a EnergyPlus IDF for sim-ulation. Executing simulations in parallel is achievedby calling the EnergyPlus RunDirMulti.bat file.

Uncertainty quantification based on data573 of the 649 parameters were assigned probabil-ity distributions based on observed data. Figure 4shows the application’s interface when fitting paramet-ric probability distributions to the data. Users shouldfirst load a comma-separated (CSV) file containing thedata (point 1 in Figure 4). An option to load a singleCSV file containing multiple datasets is also provided.The CSV file should be in a format where each columncontains the observed values for one uncertain param-eter. The first row of the file should also contain thenames of the uncertain parameters and match how theyare defined in the EnergyPlus IDF.Added functionality includes the flexibility to selectthe criterion (negative log-likelihood, BIC, AIC orAICc) for determining the probability distribution thatbest fits observed data (point 2 in Figure 4), and theoption to truncate the distribution to a lower and up-per bound (point 3 of Figure 4). The default is to trun-cate the selected distribution to the minimum and max-imum of the observed data. After fitting all parametricprobability distributions to the data, details of the dis-tribution that gives the best fit is displayed in the leftpanel (point 4 in Figure 4). A graph of the top 3 proba-bility distributions is also displayed (point 5 in Figure4). Lastly, a histogram plot of the samples generatedfrom the selected distribution is shown (point 6 in Fig-ure 4).

Schedules as random variables

Table 1Modeling schedules as random variables.

Type Hour Schedule Value

weekday

0− 1 random variable 11− 2 random variable 2...

...23− 24 random variable 24

weekend

0− 1 random variable 251− 2 random variable 26...

...23− 24 random variable 48

Schedules form an important part of building energymodeling and allows the user to influence the schedul-


- 2798 -

ing of many items (such as occupancy density, lightingand thermostatic controls) (UIUC and LBNL, 2014).To model a building in EnergyPlus users are requiredto provide a schedule value for each hour of the simu-lation period. For the case study, schedules were mod-

eled as random variables by defining each schedule tocontain a weekday and weekend part with each hourbeing defined as a random variable (Table 1). Thismeans that the schedule for each item would contain48 random variables.

Figure 3 Defining random variables in EnergyPlus IDF and loading it into the application.

Figure 4 Fit probability distributions to data and using best fit distribution to generate random values.


- 2799 -

Figure 5 Generate random values from specified probability distribution model.

Uncertainty quantification without data

Uncertain parameters without observed data but whosevalues can be obtained from specifications and draw-ings were assigned a uniform distribution with up-per and lower bounds that are ±10% of their nom-inal values respectively. The remaining parameterswere modeled based on information from literature,existing standards and past case studies. Uncertain-ties in thermal properties of materials (conductivity,density and specific heat capacity) were modeled asquantified by Macdonald (2002). Infiltration acrossthe building’s exterior surface was modeled as a trun-cated normal distribution with µ = 0.0012 m3/s.m2,σ = 0.0015m3/s.m2 and lower and upper bounds of0.00011 m3/s.m2 and 0.0069 m3/s.m2 respectively(Emmerich and Persily, 2005). These values were ob-tained by converting the values presented in Emmerichand Persily (2005) from 75Pa to 4Pa, assuming aflow exponent of 0.65. The fraction of radiant heatgiven off by people is assigned a U(0.27, 0.60) prob-ability distribution, assuming that the occupants areseated and doing light work (ASHRAE, 2009). Thefraction of radiant heat given off by lights is assigned aU(0.25, 0.37) probability distribution (Chantrasrisalaiand Fisher, 2007).Pairing every uncertain parameter with distributiontypes from literature is very time consuming. Toease this process, a database pairing various inputs inEnergyPlus to an appropriate probability distributionwas created. The database also includes the sourceof information and any assumptions that was used inthe derivation of the recommended probability model.The application reads this database and recommends

probability distributions based on the input detected.As an example, the uncertain parameter in the Energy-Plus IDF has been identified as the conductivity of amaterial (Figure 5). The application searches throughthe database and displays all probability models formaterial conductivity in the left panel. Using this ap-plication, users are able to easily create probability dis-tributions (points 2 and 3 in Figure 5) from which val-ues would then be sampled from; These values act asinputs to the building energy models. Similar to fittingdistributions to data, users are also provided with theoption to truncate the distribution (point 4 in Figure 5).

DISCUSSION AND RESULT ANALYSISNumber of simulations

1000 EnergyPlus simulations were run using the pro-posed framework and with the help of the applica-tion. Suppose that the outputs of the EnergyPlus sim-ulations X1, ..., Xn are i.i.d. with mean µ and vari-ance σ. The central limit theorem states that X =n−1

∑ni=1Xi has a distribution that is approximately

Normal with mean µ and variance σ2/n (Wasserman,2011). This means that probability statements aboutX can be approximated with a normal distribution.Hence, a normal-based confidence interval can be usedto estimate the number of simulations needed in orderto achieve reliable results.Figure 6 shows a plot of a normalized confidence in-terval ( 95% confidence interval

standard deviation ) against the total number ofsimulations. It can be observed that after 200-300 sim-ulations, additional simulations have a marginal effecton increasing confidence in the estimation of the pop-ulation mean. This suggests that the proposed method


- 2800 -

requires about 200-300 simulations.

Figure 6 Normalized confidence interval against totalnumber of simulations.

Uncertainty analysisData collection for this study took place between 16March 2013 and 18 December 2013. To allow forcomparison, the energy simulations were run over thesame period of time. Figure 7 shows a histogram of thepredicted total gas energy consumption from the 1000EnergyPlus simulations normalized by the total build-ing area. From the simulation results, the normalizedgas energy consumption has a mean of 51.2 kWh/m2

and standard deviation of 9.08 kWh/m2. From Fig-ure 7, it can be observed that the total measured gasenergy consumption (46.2 kWh/m2) falls within onestandard deviation of the mean of the predictions. Asimilar observation can be made for the total electricenergy consumption (Figure 8). From the results, thenormalized electric energy consumption has a meanof 94.6 kWh/m2 and a standard deviation of 4.67kWh/m2. Measured data (96.7 kWh/m2) also fallswithin one standard deviation from the mean of thepredictions (Figure 8).

Figure 7 Histogram of predicted total gas energyconsumption (March 2013 to December 2013).

Figure 8 Histogram of predicted total electric energyconsumption (March 2013 to December 2013).

To further analyze the simulation results, Figure 9shows a box plot of the predicted gas energy con-sumption from the 1000 EnergyPlus simulations at amonthly resolution. The gas energy consumption foreach month was normalized by dividing it by the to-tal building area. Using a box plot provides a graph-ical display of the predictions’ symmetry, skewnessand spread at a glance. In addition, outliers are alsoshown. An outlier is defined as one that falls beyondthe quartiles (25th and 75th percentile) by 1.5 timesthe inter-quartile range. It can be observed that mea-sured gas energy consumption falls within the mini-mum and maximum values predicted by the energymodels. This is with the exception of the month ofMarch where the observed value is slightly higher thanthe largest value predicted by the energy models.A similar observation can be made from Figure 10,which shows a box plot of the normalized monthlypredicted electric energy consumption and how it com-pares with measured data. It can be seen that themeasured electric energy consumption falls within therange predicted by the energy models, with Marchconsumption being slightly above the maximum valueand amongst the outliers. By providing probabilisticestimates instead of the usual point estimate with nomeasure of uncertainty, the proposed framework cancontribute to risk quantification. As a result, decisionmakers would have greater confidence in the modelssince they are provided with more information whenevaluating alternative designs and energy conservationmeasures.


- 2801 -

Figure 9 Comparison of model predictions with measured gas consumption.

Figure 10 Comparison of model predictions with measured electricity consumption.


- 2802 -

CONCLUSIONThis paper has established a structured framework formodeling uncertainties in building performance simu-lation. Through a case study and using the applicationthat was developed, 649 EnergyPlus input parameterswere modeled as uncertain. In particular, a method-ology for using observed data to quantify uncertaintywas also demonstrated. It should be noted that MonteCarlo Analysis in building performance simulation iscommonly carried out assuming a Gaussian, uniformor triangle distribution. This may not be representativeof the uncertainties in actual conditions. Generatingrandom samples from probability distributions that arebased on observed data therefore provides the addedbenefit of being able to more accurately represent un-certainties. The underlying concept of this work is thatby considering uncertainties and providing probabilis-tic predictions instead of the usual point estimates, de-cision makers would have greater confidence in simu-lation results.At present, the database has been populated with para-metric probability distributions that quantify uncer-tainties in material thermal properties, lighting powerdensities, equipment power densities, envelope infil-tration and thermal properties of occupants and lights.However, uncertainties in HVAC and other buildingsystems are still lacking. To increase the adoption ofthe proposed framework in building performance sim-ulation, more research in the quantification of theseuncertainties is required. More case studies is alsoneeded to test the robustness of the proposed methodand affirm the findings presented in this paper.It is believed that with increasing ease in the applica-tion of Monte Carlo Analysis to building energy sim-ulation programs such as EnergyPlus, its usage willbecome more widespread and commonplace.

REFERENCESASHRAE 2009. ASHRAE handbook fundamen-

tals. American Society of Heating, Refrigeratingand Air-Conditioning Engineers, Inc., Atlanta, GA,U.S.A.

Chantrasrisalai, C. and Fisher, D. E. 2007. Light-ing heat gain parameters: Experimental results (rp-1282). HVAC&R Research, 13(2):305–324.

De Wit, S. 2004. Uncertainty in building simulation.In Malkawi, A. and Augenbroe, G., editors, Ad-vanced building simulation. Taylor & Francis.

De Wit, S. and Augenbroe, G. 2002. Analysis of un-certainty in building design evaluations and its im-plications. Energy and Buildings, 34(9):951–958.

DOE 2014. Weather data for simulation.http://apps1.eere.energy.gov/buildings/energyplus/weatherdata simulation.cfm.

Eisenhower, B., O’Neill, Z., Fonoberov, V. A., andMezic, I. 2012. Uncertainty and sensitivity de-composition of building energy models. Journal ofBuilding Performance Simulation, 5(3):171–184.

Emmerich, S. J. and Persily, A. K. 2005. Airtightnessof commercial buildings in the us. In 26th AIVCConference, Brussels, Belgium.

EVO 2012. International performance measurementand verification protocol: Concepts and options fordetermining energy and water savings volume 1. Ef-ficiency Valuation Organization.

Heo, Y., Choudhary, R., and Augenbroe, G. 2012.Calibration of building energy models for retrofitanalysis under uncertainty. Energy and Buildings,47:550–560.

Hopfe, C. J. and Hensen, J. L. 2011. Uncertainty anal-ysis in building performance simulation for designsupport. Energy and Buildings, 43(10):2798–2805.

Lee, B. D., Sun, Y., Augenbroe, G., and Paredis, C. J.2013. Towards better prediction of building per-formance: A workbench to analyze uncertainty inbuilding simulation. In Proceedings of the 13thIBPSA Building Simulation Conference.

Macdonald, I. A. 2002. Quantifying the effects of un-certainty in building simulation. PhD thesis, Uni-versity of Strathclyde.

Reddy, T. A. 2011. Applied data analysis and mod-eling for energy engineers and scientists. SpringerScience & Business Media.

Sheppard, M. 2012. Fit all valid parametric probabilitydistribution to data. Matlab Central File Exchange.

Struck, C. and Hensen, J. 2007. On supporting designdecisions in conceptual design addressing specifica-tion uncertainties using performance simulation. InProceedings of the 10th IBPSA Building SimulationConference.

UIUC and LBNL 2014. Energyplus input output refer-ence: the encyclopedic reference to energyplus in-put and output. US Department of Energy.

Wasserman, L. 2011. All of statistics. Springer Sci-ence & Business Media.


- 2803 -

http://apps1.eere.energy.gov/buildings/energyplus/weatherdata_simulation.cfm

http://apps1.eere.energy.gov/buildings/energyplus/weatherdata_simulation.cfm

uncertainty analysis in building energy simulation … · uncertainty analysis in building energy...

Documents