Simulation-Based Performance Evaluation of Missing Data Handling in Network Analysis

Abstract

Network analysis has gained popularity as an approach to investigate psychological constructs. However, there are currently no guidelines for applied researchers when encountering missing values. In this simulation study, we compared the performance of a two-step EM algorithm with separated steps for missing handling and regularization, a combined direct EM algorithm, and pairwise deletion. We investigated conditions with varying network sizes, numbers of observations, missing data mechanisms, and percentages of missing values. These approaches are evaluated with regard to recovering population networks in terms of loss in the precision matrix, edge set identification and network statistics. The simulation showed adequate performance only in conditions with large samples ($n\ge 500$) or small networks (p = 10). Comparing the missing data approaches, the direct EM appears to be more sensitive and superior in nearly all chosen conditions. The two-step EM yields better results when the ratio of n/p is very large – being less sensitive but more specific. Pairwise deletion failed to converge across numerous conditions and yielded inferior results overall. Overall, direct EM is recommended in most cases, as it is able to mitigate the impact of missing data quite well, while modifications to two-step EM could improve its performance.

Keywords:

• Network analysis
• missing values
• simulation study
• graphical lasso regularization
• EM algorithms

Article information

Conflict of Interest Disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical Principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This work was not supported by any funding.

Role of the Funders/Sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The authors would like to thank Simon Grund for his support in the implementation of the amputation processes. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institution is not intended and should not be inferred.

Notes

1 This can be requested by setting missing =“fiml” in the estimateNetwork-function of bootnet.

2 This coefficient is commonly used in studies on network analysis and is identical to the correlation coefficient $\varphi ,$ which is widely used in psychology. Both are also identical to the product-moment correlation of two dichotomous variables.

3 For readers with a working distribution of RStudio, the App can be started directly using source(”https://osf.io/y9f56/download/”). The corresponding code and data can be found on the Open Science Framework (https://osf.io/y9f56/). For readers without RStudio, the App is also hosted on https://nehler.shinyapps.io/Nehler_Schultze_Missings_in_network_analysis/. The app also offers some statistics not included in the paper such as betweenness and density.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.