Uncertainty and sensitivity analysis of building-stock energy models: sampling procedure, stock size and Sobol’ convergence

References

• Aronhime, S., C. Calcagno, G. H. Jajamovich, H. A. Dyvorne, P. Robson, D. Dieterich, … B. Taouli. 2014. “DCE-MRI of the Liver: Effect of Linear and Nonlinear Conversions on Hepatic Perfusion Quantification and Reproducibility.” Journal of Magnetic Resonance Imaging 40 (1): 90–98. doi:10.1002/jmri.24341
   PubMed Web of Science ®Google Scholar
• Booth, A. T., R. Choudhary, and D. J. Spiegelhalter. 2012. “Handling Uncertainty in Housing Stock Models.” Building and Environment 48: 35–47. doi:10.1016/j.buildenv.2011.08.016
   Web of Science ®Google Scholar
• Bracke, W., M. Delghust, J. Laverge, J. Alderweireldt, C. Leveau, F. Pierard, L. Peeters, and J. Beaten. 2015. Studie EPB-eisenpakket residentiële nieuwbouw: een nieuwe indicator voor de energieprestatie indicator op weg naar 2021. https://assets.vlaanderen.be/image/upload/v1662982456/EPB-eisen_-_EPB-eisenpakket_residenti%C3%ABle_nieuwbouw_Studie_kiluab.pdf.
   Google Scholar
• Bracke, W., M. Delghust, J. Laverge, and A. Janssens. 2018. “Building Energy Performance: Sphere Area as a Fair Nomalisation Concept.” Building Research & Information 47 (5): 549–566. doi:10.1080/09613218.2018.1467703
   Web of Science ®Google Scholar
• Campolongo, F., and A. Saltelli. 1997. “Sensitivity Analysis of an Environmental Model: An Application of Different Analysis Methods.” Reliability Engineering & System Safety 57 (1): 49–69. doi:10.1016/S0951-8320(97)00021-5
   Web of Science ®Google Scholar
• Cerezo Davila, C. 2017. “Building Archetype Calibration for Effective Urban Building Energy Modelling.” Ph.D. thesis, Massachusetts Institute of Technology.
   Google Scholar
• Cozza, S., J. Chambers, and M. K. Patel. 2020. “Measuring the Thermal Energy Performance gap of Labelled Residential Buildings in Switzerland.” Energy Policy 137: 2020. doi:10.1016/j.enpol.2019.111085
   Web of Science ®Google Scholar
• Dascalaki, E. G., K. G. Droutsa, C. A. Balaras, and S. Kontoyiannidis. 2011. “Building Typologies as Atool for Assessing the Energy Performance of Residential Buildings – a Case Study for the Hellenic Building Stock.” Energy & Buildings 43: 3400–3409. doi:10.1016/j.enbuild.2011.09.002
   Web of Science ®Google Scholar
• Dawid, A. P. 1984. “Statistical Theory: The Prequential Approach.” Journal of the Royal Statistical Society 147: 278–292. doi:10.2307/2981683
   Google Scholar
• Dawid, A. P. 1985. “Calibration-based Empirical Probability.” The Annals of Statistics 13 (4): 1251–1273. doi:10.1214/aos/1176349736
   Web of Science ®Google Scholar
• DB. 2021. Zusätzliches Geld für den Klimaschutz. Berlin: Die Bundesregierung.
   Google Scholar
• Delghust, M. 2015a. “Improving the Predictive Power of Simplified Residential Space Heating Demand Models: A Field Data and Model Driven Study.” PhD disser., Ghent University.
   Google Scholar
• Delghust, M., Y. De Weerdt, and A. Janssens. 2015b. “Zoning and Intermittency Simplifications in Quasi-Steady State Models.” Proceedings of the 6th International Building Physics Conference (IBPC2015). Torino, Italy.
   Google Scholar
• Delghust, M., T. Strobbe, R. De Meyer, and A. Janssens. 2015c. “Enrichment of Single-zone EPB-data into Multi-zone Models using BIM-based Parametric Typologies.” Proceedings of the 14th International Conference of IBPSA (BS2015). Hyderabad, India.
   Google Scholar
• DIN. 2007. DIN V 18599-2:2007-02 Energy Efficiency of Buildings — Calculation of the Energy Needs, Delivered Energy and Primary Energy for Heating, Cooling, Ventilation, Domestic Hot Water and Lighting. Deutschen Instituts für Normung.
   Google Scholar
• EC. 2020. “JRC Technical Report: Uncertainty and Sensitivity Analysis for Policy Decision Making.” European Commission.
   Google Scholar
• Eisenhower, B., Z. O’Neill, V. A. Fonoberov, and I. Mezic. 2012. “Uncertainty and Sensitivity Decomposition of Building Energy Models.” Journal of Building Performance Simulation 5 (3): 171–184. doi:10.1080/19401493.2010.549964
   Web of Science ®Google Scholar
• EU. 2019. The European Parliament Declares Climate Emergency. European Parliament.
   Google Scholar
• EU. 2020a. Climate Target Plan. European Commission.
   Google Scholar
• EU. 2020b. Stepping up Europe’s 2030 Climate Ambition: Investing in a Climate-Neutral Future for the Benefit of our People. European Commission.
   Google Scholar
• EU. 2020c. Energy Performance of Buildings Directive. European Commission Department of Energy.
   Google Scholar
• EU. 2021a. Climate Action: Citizen Support for Climate Action. European Commission.
   Google Scholar
• EU. 2021b. EU Climate Action Policy: Responding to the Global Emergency. European Parliament Think Tank .
   Google Scholar
• Fennell, P. J., S. Ebrahimigharehbaghi, E. Mata, G. Kokogiannakis, S. Amrith, and S. Ignatiadou. 2023. “A Review of the Status of Uncertainty and Sensitivity Analysis in Building-Stock Energy Models.” Renewable and Sustainable Energy Reviews (in review).
   Web of Science ®Google Scholar
• Fennell, P. J., P. A. Ruyssevelt, E. Mata, and M. Jacob. 2019. “A Review of the Status of Uncertainty and Sensitivity Analysis in Building-stock Energy Models.” Proceedings of the 16th International Conference of IBPSA (BS2019). Rome, Italy.
   Google Scholar
• Fennell, P. J., M. Van Hove, L. Weinberg, G. J. Bennett, M. Delghust, S. Forthuber, M. Jakob, et al. 2021. “Challenges and Lessons Learned in Applying Sensitivity Analysis to Building Stock Energy Models.” Proceedings of the 17th International Conference of IBPSA (BS2021). Bruges, Belgium.
   Google Scholar
• Ferretti, F., A. Saltelli, and S. Tarantola. 2016. “Trends in Sensitivity Analysis Practice in the Last Decade.” Science of the Total Environment 568: 666–670. doi:10.1016/j.scitotenv.2016.02.133
   PubMed Web of Science ®Google Scholar
• FMRA. 2019. Integrated National Energy and Climate Plan for Austria. Federal Ministry Republic of Austria.
   Google Scholar
• Fracastoro, G. V., and M. Serraino. 2011. “A Methodology for Assessing the Energy Performance of Large Scale Building Stocks and Possible Applications.” Energy and Buildings 43 (4): 844–852. doi:10.1016/j.enbuild.2010.12.004
   Web of Science ®Google Scholar
• FRG. 2017. Climate Plan. France Gouvernement.
   Google Scholar
• Gorris, L. G. M., and C. Yoe. 2014. “Risk Analysis: Risk Assessment: Principles, Methods, and Applications.” Encyclopedia of Food Safety 1: 65–72. doi:10.1016/B978-0-12-378612-8.00031-7
   Google Scholar
• Grassie, D., I. Korolija, D. Mumovic, and P. A. Ruyssevelt. 2018. “Feedback and Feedforward Mechanisms for Generating Occupant Datasets for UK School Stock Simulation Modelling.” Proceedings of Building Simulation and Optimization.
   Google Scholar
• Guerra-Santin, O., L. Itard, and H. Visscher. 2009. “The Effect of Occupancy and Building Characteristics on Energy use for Space and Water Heating in Dutch Residential Stock.” Energy and Buildings 41 (11): 1223–1232. doi:10.1016/j.enbuild.2009.07.002
   Web of Science ®Google Scholar
• Hamilton, I. G., A. J. Summerfield, R. Lowe, P. Ruyssevelt, C. A. Elwell, and T. Oreszczyn. 2013. “Energy Epidemiology: A new Approach to end-use Energy Demand Research.” Building Research & Information 41 (4): 482–497. doi:10.1080/09613218.2013.798142
   Web of Science ®Google Scholar
• Herman, J., and W. Usher. 2017. “SALib: An Open-Source Python Library for Sensitivity Analysis.” Journal of Open Source Software 2 (9): 97. doi:10.21105/joss.00097
   Google Scholar
• Hodges, J. S. 1987, August. “Uncertainty, Policy Analysis, and Statistics.” Statistical Science 2(3): 259–275. doi:10.1214/ss/1177013224
   Google Scholar
• Hopfe, C. 2009. “Uncertainty and Sensitivity Analysis in Building Performance Simulation for Decision Support and Design Optimisation.” PhD Thesis. Technische Universiteit Eindhoven.
   Google Scholar
• IEA. 2019a. Global Status Report for Buildings and Construction 2019: Towards a Zero-Emissions, Efficient and Resilient Buildings and Construction Sector. International Energy Agency.
   Google Scholar
• IEA. 2019b. “Energy Efficiency: Buildings. The Global Exchange for Energy Efficiency Policies, Data and Analysis.” https://www.iea.org/topics/energyefficiency/buildings/.
   Google Scholar
• IEA EBC Annex 70. 2017. “Building Energy Epidemiology.” https://energyepidemiology.org/.
   Google Scholar
• ISO. 2007. ISO 13790:2007(E) Energy Performance of Buildings. Calculation of Energy use for Space Heating and Cooling. Geneva: International Organisation for Standardisation (ISO).
   Google Scholar
• Kavgic, M., A. Mavrogianni, D. Mumovic, A. Summerfield, Z. Stevanovic, and M. Djurovic-Petrovic. 2010. “A Review of Bottom-up Building Stock Models for Energy Consumption in the Residential Sector.” Building and Environment 45: 1683–1697. doi:10.1016/j.buildenv.2010.01.021
   Web of Science ®Google Scholar
• Langevin, J., J. L. Reyna, S. Ebrahimigharebhaghi, N. Holck Sandberg, P. Fennell, C. Nägeli, J. Laverge, M. Delghust, M. Van Hove, J. Webster, et al. 2020. “Developing a Common Approach for Classifying Building Stock Energy Models.” Renewable & Sustainable Energy Reviews 133. doi:10.1016/j.rser.2020.110276
   Web of Science ®Google Scholar
• Lim, H., and Z. J. Zhai. 2017. “Review on Stochastic Modeling Methods for Building Stock Energy Prediction.” Building Simulation 10: 607–624. doi:10.1007/s12273-017-0383-y
   Web of Science ®Google Scholar
• Loucks, D. P., and E. van Beek. 2017. Water Resource Systems Planning and Management: An Introduction to Methods, Models, and Applications. Spinger: Deltares and UNESCO-IHE. Doi: 10.1007/978-3-319-44234-1
   Google Scholar
• Macdonald, I. 2002. “Quantifying the Effects of Uncertainty in Building Simulation.” PhD thesis, University of Strathclyde.
   Google Scholar
• Majcen, D., L. C. M. Itard, and H. Visscher. 2013. “Theoretical vs.Actual Energy use of Labelled Dwellings in the Netherlands: Discrepancies and Policy Implications.” Energy Policy 54: 125–136. doi:10.1016/j.enpol.2012.11.008
   Web of Science ®Google Scholar
• Mata, É, A. Sasic Kalagasidis, and F. Johnsson. 2013. “Energy Usage and Technical Potential for Energy Saving Measures in the Swedish Residential Building Stock.” Energy Policy 55: 404–414. doi:10.1016/j.enpol.2012.12.023
   Web of Science ®Google Scholar
• Mata, É, A. Sasic Kalagasidis, and F. Johnsson. 2014. “Building-stock Aggregation Through Archetype Buildings: France, Germany, Spain and the UK.” Building and Environment 81: 270–282. doi:10.1016/j.buildenv.2014.06.013
   Web of Science ®Google Scholar
• Mathew, P. A., L. N. Dunn, M. D. Sohn, A. Mercado, C. Custudio, and T. Walter. 2015. “Big-data for Building Energy Performance: Lessons from Assembling a Very Large National Database of Building Energy use.” Applied Energy 140: 85–93. doi:10.1016/j.apenergy.2014.11.042
   Web of Science ®Google Scholar
• Menberg, K., Y. Heo, and R. Choudhary. 2016. “Sensitivity Analysis Methods for Building Energy Models: Comparing Computational Costs and Extractable Information.” Energy and Buildings 133: 433–445. doi:10.1016/j.enbuild.2016.10.005
   Web of Science ®Google Scholar
• Morris, M. D. 1991. “Factorial Sampling Plans for Preliminary Computational Experiments.” Technometrics 33: 161–174. doi:10.1080/00401706.1991.10484804
   Web of Science ®Google Scholar
• Müller, A. 2015. “Energy Demand Assessment for Space Conditioning and Domestic Hot Water: A Case Study for the Austrian Building Stock.” PhD Thesis. Technische Universität Wien.
   Google Scholar
• Naber, E., R. Volk, and F. Schultmann. 2017. “From the Building Level Energy Performance Assessment to the National Level: How are Uncertainties Handled in Building Stock Models.” Procedia Engineering, 1443–1452. doi:10.1016/j.proeng.2017.04.307
   Google Scholar
• Nägeli, C., C. Camarasa, M. Jakob, G. Catenazzi, and Y. Ostermeyer. 2018. “Synthetic Building Stocks as a Way to Assess the Energy Demand and Greenhouse Gas Emissions of National Building Stocks.” Energy and Buildings 173. doi:10.1016/j.enbuild.2018.05.055
   Web of Science ®Google Scholar
• NEKP. 2019. Nationaal Energie- en Klimaatplan 2021 - 2030. NEKP.
   Google Scholar
• NEN. 2012. NEN 7120 + C Energieprestatie van gebouwen - Bepalingsmethode. NEderlandse Norm.
   Google Scholar
• Nguyen, A. T., and S. Reiter. 2015. “A Performance Comparison of Sensitivity Analysis Methods for Building Energy Models.” Building Simulation 8 (6): 651–664. doi:10.1007/s12273-015-0245-4
   Web of Science ®Google Scholar
• Nossent, J., P. Elsen, and W. Bauwens. 2011. “Sobol’ Sensitivity Analysis of a Complex Environmental Model.” Environmental Modelling & Software 26: 1515–1525. doi:10.1016/j.envsoft.2011.08.010
   Web of Science ®Google Scholar
• Österbring, M., É Mata, L. Thuvander, M. Mangold, F. Johnsson, and H. Wallbaum. 2016. “A Differentiated Description of Building-Stocks for a Georeferenced Urban Bottom-up Building-Stock Model.” Energy and Buildings 120: 78–84. doi:10.1016/j.enbuild.2016.03.060
   Web of Science ®Google Scholar
• Pérez-Lombard, L., J. Ortiz, R. González, and I. R. Maestre. 2009. “A Review of Benchmarking, Rating and Labelling Concepts Within the Framework of Building Energy Certification Schemes.” Energy and Buildings 41: 272–278. doi:10.1016/j.enbuild.2008.10.004
   Web of Science ®Google Scholar
• Pichery, C. 2014. “Sensitivity Analysis.” In Encyclopedia of Toxicology, (3rd ed., 236–237). doi:10.1016/B978-0-12-386454-3.00431-0
   Google Scholar
• Reddy, T. A., I. Maor, and C. Panjapornpon. 2007. “Calibrating Detailed Building Energy Simulation Programs with Measured Data - Part I: General Methodology.” HVAC&R Research 13 (2): 221–241. doi:10.1080/10789669.2007.10390952
   Web of Science ®Google Scholar
• Refsgaard, J. C., and H. J. Henriksen. 2004. “Modelling Guidelines: Terminology and Guiding Principles.” Advances in Water Resources 27: 71–82. doi:10.1016/j.advwatres.2003.08.006
   Web of Science ®Google Scholar
• Refsgaard, J. C., J. P. van der Sluijs, A. L. Hojberg, and P. A. Vanrolleghem. 2007. “Uncertainty in the Environmental Modelling Process: A Framework and Guidance.” Environmental Modelling & Software 22: 1543–1556. doi:10.1016/j.envsoft.2007.02.004
   Web of Science ®Google Scholar
• Reinhart, C. F., and C. C. Davila. 2016. “Urban Building Energy Modeling - a Review of a Nascent Field.” Building and Environment 97: 196–202. doi:10.1016/j.buildenv.2015.12.001
   Web of Science ®Google Scholar
• Reyna, J. L., and M. V. Chester. 2017. “Energy Efficiency to Reduce Residential Electricity and Natural gas use Under Climate Change.” Nature Communications 8: 14916. doi:10.1038/ncomms14916
   PubMed Web of Science ®Google Scholar
• Reyna, J., E. Wilson, A. Parker, A. Satre-Meloy, A. Egerter, C. Bianchi, M. Praprost, et al. 2022. “U.S. Building Stock Characterization Study: A National Typology for Decarbonizing U.S. Buildings.” NREL (Technical Report).
   Google Scholar
• Saltelli, A., K. Aleksankina, W. Becker, P. Fennell, F. Ferretti, N. Holst, S. Li, and Q. Wu. 2019. “Why so Many Published Sensitivity Analyses are False: A Systematic Review of Sensitivity Analysis Practices.” Environmental Modelling & Software 114: 29–39. doi:10.1016/j.envsoft.2019.01.012
   Web of Science ®Google Scholar
• Saltelli, A., and P. Annoni. 2010. “How to Avoid a Perfunctory Sensitivity Analysis.” Environmental Modelling & Software 25 (12): 1508–1517. doi:10.1016/j.envsoft.2010.04.012
   Web of Science ®Google Scholar
• Saltelli, A., M. Ratto, T. Andres, F. Campolongo, J. Cariboni, D. Gatelli, M. Saisana, and S. Tarantola. 2008. Global Sensitivity Analysis. The Primer. Chichester, West Sussex: John Wiley and Sons. doi:10.1002/9780470725184.ch6
   Google Scholar
• Saltelli, A., S. Tarantola, and F. Campolongo. 2000. “Sensitivity Analysis as an Ingredient of Modeling.” Statistical Science 15 (4): 377–395. http://www.jstor.org/stable/2676831.
   Web of Science ®Google Scholar
• Saltelli, A., S. Tarantola, F. Campolongo, and M. Ratto. 2004. Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models (Vol. 1). Probability and Statistics. Chichester, NY:  John Wiley & Sons.
   Google Scholar
• Saltelli, A., S. Tarantola, and K. Chan. 1999. “Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output.” Technometrics 41 (1): 39–56. doi:10.1080/00401706.1999.10485594
   Web of Science ®Google Scholar
• Sandberg, N. H., I. Sartori, and H. Brattebø. 2014a. “Using a Dynamic Segmented Model to Examine Future Renovation Activities in the Norwegian Dwelling Stock.” Energy and Buildings 82: 287–295. doi:10.1016/j.enbuild.2014.07.005
   Web of Science ®Google Scholar
• Sandberg, N. H., I. Sartori, and H. Brattebø. 2014b. “Sensitivity Analysis in Long-Term Dynamic Building Stock Modelling - Exploring the Importance of Uncertainty of Input Parameters in Norwegian Segmented Dwelling Stock Model.” Energy and Buildings 85: 136–144. doi:10.1016/j.enbuild.2014.09.028
   Web of Science ®Google Scholar
• Santner, T. J., B. J. Williams, and W. I. Notz. 2003. Design and Analysis of Computer Experiments. New York: Springer-Verlag.
   Google Scholar
• Sarrazin, F., F. Pianosi, and T. Wagener. 2016. “Global Sensitivity Analysis of Environmental Models: Convergence and Validation.” Environmental Modelling & Software 79: 135–152. doi:10.1016/j.envsoft.2016.02.005
   Web of Science ®Google Scholar
• Sobol’, I. M. 1990. “Sensitivity Estimates for Nonlinear Mathematical Models.” Maticheskoe Modelirovanie 2: 112–118. (in Russian), translated in English (1993). In: Mathematical Modelling and Computational Experiments. 1, 407-414.
   Google Scholar
• Sobol, I., and A. Gresham. 1995. “On an Alternative Global Sensitivity Estimators.” Proceedings of SAMO, Belgirate, 40–42.
   Google Scholar
• Song, X., J. Zhang, C. Zhan, Y. Xuan, M. Ye, and C. Xu. 2015. “Global Sensitivity Analysis in Hydrological Modelling: Review of Concepts, Methods, Theoretical Framework, and Applications.” Journal of Hydrology 523: 739–757. doi:10.1016/j.jhydrol.2015.02.013
   Web of Science ®Google Scholar
• Sousa, G., B. M. Jones, P. A. Mirzaei, and D. Robinson. 2017. “A Review and Critique of UK Housing Stock Energy Models, Modelling Approaches and Data Sources.” Energy and Buildings 151: 66–80. doi:10.1016/j.enbuild.2017.06.043
   Web of Science ®Google Scholar
• Strobbe, T., R. Verstraeten, M. Delghust, J. Laverge, R. De Meyer, and A. Janssens. 2015. “Using a Building Information Modelling Approach for Teaching About Residential Energy Use and Official Energy Performance.” 14th International Conference of IBPSA (BS2015). Hyderabad, India.
   Google Scholar
• Sustainability Solution Group. 2021. “A State of the Art City-Scale Energy, Emissions and Finance Model.” http://cityinsight.ssg.coop/.
   Google Scholar
• Tian, W. 2013. “A Review of Sensitivity Analysis Methods in Building Energy Analysis.” Renewable and Sustainable Energy Reviews 20: 411–419. doi:10.1016/j.rser.2012.12.014
   Web of Science ®Google Scholar
• Van Hove, M. Y. C., M. Delghust, and A. Janssens. 2021. Analyse naar de haalbaarheid van statistische modellen die energiegebruik in woningen kunnen voorspellen op basis van gebouwparameters. https://www.energiesparenbe/marktonderzoek.
   Google Scholar
• Van Hove, M. Y. C., M. Deurinck, W. Lameire, J. Laverge, A. Janssens, and M. Delghust. 2022. “Large-scale Statistical Analysis and Modelling of Real and Regulatory Total Energy use in Existing Single-Family Houses in Flanders.” Building Research and Information 51 (2): 203–222. doi:10.1080/09613218.2022.2113023
   Web of Science ®Google Scholar
• VEKA. 2015. Open Data in Vlaanderen. Flemish Energy and Climate Agency. https://opendata.vlaanderen.be/
   Google Scholar
• VEKA. 2019. “EPB-Bijlage V: Bepalingsmethode van het Peil van Primair Energieverbruik van Woongebouwen.” In  Belgisch Staatsblad - Moniteur Belge, edited by VEKA, 1–216. Brussels: Flemish Regional Government.
   Google Scholar
• Walker, W. E., P. Harremoës, J. Rotmans, J. P. Van Der Sluijs, M. B. Van Asselt, P. Janssen, and M. P. Krayer von Krauss. 2003. “Defining Uncertainty: A Conceptual Basis for Uncertainty Management in Model-Based Decision Support.” Integrated Assess 4 (1): 5–17. doi:10.1076/iaij.4.1.5.16466
   Google Scholar
• Wilson, E. 2017. “ResStock - Targeting Energy and Cost Savings for U.S. Homes.” https://www.nrel.gov/docs/fy17osti/68653.pdf.
   Google Scholar
• Wilson, E. J., C. B. Christensen, S. G. Horowitz, J. J. Robertson, and J. B. Maguire. 2017. Energy Efficiency Potential in the U.S. Single-Family Housing Stock. National Renewable Energy Lab (NREL).
   Google Scholar