An evaluation of the genome-wide false positive rates of common methods for identifying differentially methylated regions using illumina methylation arrays

Figures & data

Table 1. Summary of DMR methods.

Method	First Author	Year of Published	PMID	Platform /Package	Classification	Pre-defined Regions
Bumphunter	Jaffe, A.E.	2012	22,422,453	R/minfi	Supervised
comb-p	Pedersen, B.S.,	2012	22,954,632	Python/comb-p	Supervised
DMRcate	Peters, T.J.	2015	25,972,926	R/DMRcate	Supervised
mCSEA	Martorell-Marugan, J.	2019	30,753,302	R/mCSEA	Unsupervised	Promoters, Gene bodies, CGI
coMethDMR	Gomez, L.	2019	31,291,459	R/coMethDMR	Unsupervised	Genic, Intergenic

Table 2. Summary of DMR Implementations.

Methods	Software	Function	Parameter Settings
Bumphunter	R/3.6.2+	bumphunter()	maxGap = 500, pickCutoff = TRUE, pickCutoffQ = 0.99, nullMethod = ‘bootstrap,’ B = 1000, smooth = T, smoothFunction = loessByCluster
BumphunterNS	R/3.6.2+	bumphunter()	maxGap = 500, pickCutoff = TRUE, pickCutoffQ = 0.99, nullMethod = ‘bootstrap,’ B = 1000, smooth = F
comb-p	python3/3.7.7	comb-p	–seed 1e-3 – dist 200 – region-filter-p 0.1
DMRcate	R/3.6.2+	cpg.annotate()	analysis.type = ‘differential,’ fdr = 0.05
mCSEA	R/3.6.2+	mCSEATest()	regionsTypes = c(‘promoters,’ ‘genes,’ ‘CGI’)
coMethDMR	R/3.6.2+	CoMethAllRegions()	rDropThresh_num = 0.4, minCpGs = 3
		lmmTestAllRegions()	modelType = ‘randCoef’

* R/3.6.2+: R version 3.6.2 or above.

Table 3. Summary of Relative Computation Burden.

		450 K-Mono		EPIC-WB
Methods	Functions	Peak Memory	Time	Peak Memory	Time
Bumphunter	bumphunter()	111.02 GB	16.79 hours	129.76 GB	26.01 hours
BumphunterNS	bumphunter()	37.86 GB	1.08 hours	52.01 GB	3.17 hours
comb-p	comb-p	5.54 GB	<2 mins	5.60 GB	<3 mins
DMRcate	cpg.annotate()	24.96 GB	<1 mins	19.94 GB	<6 mins
mCSEA	mCSEATest()	25.12 GB	<2 mins	18.42 GB	<3 mins
coMethDMR	CoMethAllRegions() lmmTestAllRegions() AnnotateResults()	28.07 GB	1.94 hours	16.79 GB	10.71 hours

*450 K-Mono: 450 K monocytes data; EPIC-WB: EPIC whole blood data.

*Interpret results relatively. The computational estimates were computed by the shared compute nodes at Boston University Shared Computing Cluster.

Figure 1. The Genome-wide False Positive Rates on 450 K-Mono: a. Normal Phenotype, b. Dichotomous Phenotype with 50% Cases, C. Skewed Continuous Phenotype.

*Genome-wide FP: genome-wide false positive rate.*450 K-Mono: 450 K monocytes data. *Bumphunter was not available due to computational limits. *Bumphunter_NS: Bumphunter with no smoothing function.

Figure 2. The Genome-wide False Positive Rates on EPIC-WB: a. Normal Phenotype, b. Dichotomous Phenotype with 50% Cases, C. Skewed Continuous Phenotype.

*Genome-wide FP: genome-wide false positive rate.*EPIC-WB: EPIC whole blood data. *Bumphunter was not available due to computational limits. *Bumphunter_NS: Bumphunter with no smoothing function.

Figure 3. The Genome-wide False Positive Rates on 450 K-WB: a. Normal Phenotype, b. Dichotomous Phenotype with 50% Cases, C. Skewed Continuous Phenotype.

*Genome-wide FP: genome-wide false positive rate. *450 K-WB: 450 K whole blood data. *Bumphunter was not available due to computational limits. *Bumphunter_NS: Bumphunter with no smoothing function.

Figure 4. Regional False Positive Rates vs Average Absolute Pairwise Correlations on 450 K-Mono and EPIC-WB for the Normal Phenotype with Overall Trend Indicated by a Loess Curve in Red: A. coMethDMR, B. mCSEA, C. Bumphunter, D. comb-p.

*Regional FP: regional false positive rate. *AAC: average absolute pairwise correlation. *450 K-Mono: 450 K monocytes data; EPIC-WB: EPIC whole blood data.

Data Availability Statement

The GEO data GSE56046 that support the findings of this study are openly available in the MESA Epigenomics and Transcriptomics Study at https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE56046.

Qualified investigators can apply to the PTSD Genetics (NCPTSD) and TRACTS data repositories to gain access to these data via a Data Use Agreement. Please contact Drs. Miller and McGlinchey regarding access to PTSD Genetics and TRACTS data repositories, respectively.