Sensitivity and specificity evaluation of multiple neurodegenerative proteins for Creutzfeldt-Jakob disease diagnosis using a deep-learning approach

Figures & data

Table 1. Patients’ classification and clinical features of the definite, probable, possible CJD cases and other neuronal diseases patients used in this study.

Groups	No. of patients	Diagnostic criteria/Symptoms
Definite CJD	5	Diagnosed by neuropathological techniques, immunohistochemcal detection of PrP^Sc of autopsied or biopsied brain tissues from patients with probable or possible sCJD
Probable CJD	44	Rapid progressive dementia of less than 2 years’ duration and at least two of the following features such as myoclonus, visual or cerebellar disturbance, pyramidal, extrapyramidal dysfunction, akinetic mutism, with a EEG data, and/or a positive result of 14–3-3 assay
Possible CJD	11	Progressive dementia and at least two of the following features such as myoclonus, visual or cerebellar disturbance, pyramidal, extrapyramidal dysfunction, akinetic mutism, without a EEG data, and/or a positive result of 14–3-3 assay
Non-CJD	256	Cases referred to as ‘‘suspected CJD” to the KNIH, but were not fulfilling the criteria of probable or possible CJD
Neuronal diseases (test set_B)	3	Hydrocephalus, unspecified
	4	Normal Pressure Hydrocephalus
	1	Multi-system degeneration
	1	Alzheimer’s Disease
	1	Mild cognitive disorder
	1	Dementia
	1	Spinal muscular atrophy, and related syndromes
	1	Cerebral palsy
	2	Other disorders of brain

Table 2. Diagnostic performance of various cerebrospinal fluid markers for Creutzfeldt-Jakob disease according to defined criteria.

		Sensitivity % (TP/all)		Specificity % (TN/all)
Markers	Criteria	Definite and probable	Possible	Non-CJD	Neuronal diseases	References
14–3-3	14–3-3 positive^a	67.35 (33/49)	54.54 (6/11)	67.58 (173/256)	N/A	^[Citation52]
t-tau	> 1,000 pg/mL	59.18 (29/49)	72.73 (8/11)	77.34 (198/256)	86.67 (13/15)	^[Citation49]
t-tau	> 1,300 pg/mL	53.06 (26/49)	72.73 (8/11)	80.86 (207/256)	86.67 (13/15)	^[Citation50]
t-tau with p/t-tau ratio	t-tau > 1,000 pg/ml with p/t tau ratio < 0.04	55.1 (27/49)	72.73 (8/11)	79.3 (203/256)	93.33 (14/15)	^[Citation49]
Aβ	< 445 pg/mL	69.39 (34/49)	81.82 (9/11)	28.91 (74/256)	N/A	^[Citation51]
a-syn	> 820 pg/mL	55.1 (27/49)	27.27 (3/11)	78.91 (202/256)	93.33 (14/15)	^[Citation10]
a-syn	> 680 pg/mL	55.1 (27/49)	27.27 (3/11)	74.61 (191/256)	86.67 (13/15)	^[Citation17]

^aWeak positive result considered as negative

N/A: not analysed

Figure 1. Heat map of the correlation of t-tau, p-tau, p/t-tau ratio, Aβ, a-syn and 14–3-3 levels with patients. The colour of each square depicts the correlation level, ranging from black (negative correlation) to red (intermediate correlation value) to white (positive correlation value).

Figure 2. Discrimination plot for discrimination between CJD (orange) and non-CJD (blue) patients based on values of six CSF biomarkers.

Table 3. Evaluation of classifier performances (test set_A, 12 of CJD cases and 40 of non-CJD patients).

Classifier	Accuracy	Sensitivity	Specificity	AUC
SVM	76.92% (40/52)	66.67% (8/12)	80% (32/40)	0.73
Decision tree (J48)	78.85% (41/52)	41.67% (5/12)	90% (36/40)	0.8
Naïve Bayes	76.92% (40/52)	66.67% (8/12)	80% (32/40)	0.72
Random Forest	78.85% (41/52)	33.33% (4/12)	92.5% (37/40)	0.82
DNN	86.54%(45/52)	83.33%(10/12)	87.5% (35/40)	0.90

Accuracy: TP+TN/No. of cases in test set_A

Sensitivity: TP/No. of CJD cases in test set_A

Specificity: TN/No. of non-CJD cases in test set_A

Figure 3. Deep neural network showing the best performance for CJD discrimination. Our network structure consisted of one input, five hidden, and one output layer. The five hidden layers consisted of 20, 40, 30, 20, and 12 hidden unit numbers of each layer, respectively. The last two output units were used to distinguish between CJD and non-CJD patients.

Table 4. Analysis of two- and three-protein combinations for discrimination between CJD and non-CJD patients. All values were calculated using the oversampled validation set and test set_A.

Combinations	Accuracy in validation set	AUC in validation set	Accuracy in test set_A	AUC in test set_A
t-tau and p-tau	70.73%	0.83	69.23%	0.71
t-tau and p/t-tau ratio	65.85%	0.77	73.08%	0.76
t-tau and Aβ	80.49%	0.91	75%	0.76
t-tau and a-syn	80.49%	0.91	82.69%	0.86
t-tau and 14–3-3	63.41%	0.80	69.23%	0.79
p-tau and p/t-tau ratio	65.85%	0.83	61.54%	0.58
p-tau and Aβ	51.22%	0.51	46.15%	0.55
p-tau and a-syn	58.54%	0.72	65.38%	0.84
p-tau and 14–3-3	65.85%	0.71	65.38%	0.60
p/t-tau ratio and Aβ	48.78%	0.46	48.08%	0.52
p/t-tau ratio and a-syn	53.66%	0.62	59.62%	0.84
p/t-tau ratio and 14–3-3	68.29%	0.77	69.23%	0.75
Aβ and a-syn	65.85%	0.76	69.23%	0.74
Aβ and 14–3-3	56.1%	0.76	38.46%	0.42
a-syn and 14–3-3	60.98%	0.69	75%	0.90
t-tau, p-tau and p/t-tau ratio	85.21%	0.85	78.85%	0.85
t-tau, p-tau and Aβ	78.05%	0.88	75%	0.74
t-tau, p-tau and a-syn	87.8%	0.95	90.38%	0.88
t-tau, p-tau and 14–3-3	78.05%	0.91	71.15%	0.71
t-tau, p/t-tau ratio and Aβ	75.61%	0.85	80.77%	0.77
t-tau, p/t-tau ratio and a-syn	75.61%	0.90	75%	0.84
t-tau, p/t-tau ratio and 14–3-3	70.73%	0.82	76.92%	0.78
t-tau, Aβ and a-syn	78.05%	0.85	86.54%	0.86
t-tau, Aβ and 14–3-3	87.8%	0.95	78.85%	0.75
t-tau, a-syn and 14–3-3	87.8%	0.98	78.85%	0.86
p-tau, p/t-tau ratio and Aβ	46.34%	0.54	51.92%	0.60
p-tau, p/t-tau ratio and a-syn	68.29%	0.72	71.15%	0.77
p-tau, p/t-tau ratio and 14–3-3	70.73%	0.84	71.15%	0.65
p-tau, Aβ and a-syn	73.17%	0.88	84.62%	0.91
p-tau, Aβ and 14–3-3	58.54%	0.75	40.38%	0.44
p-tau, a-syn and 14–3-3	65.85%	0.70	71.15%	0.74
p/t-tau ratio, Aβ and a-syn	65.85%	0.83	69.23%	0.72
p/t-tau ratio, Aβ and 14–3-3	56.1%	0.55	26.92%	0.40
p/t-tau ratio, a-syn and 14–3-3	60.98%	0.62	71.15%	0.82
Aβ, a-syn and 14–3-3	63.41%	0.76	67.31%	0.75

Accuracy: TP+TN/No. of cases in test set_A

Sensitivity: TP/No. of CJD cases in test set_A

Specificity: TN/No. of non-CJD cases in test set_A

Muayqil T, Gronseth G, Camicioli R. Evidence-based guideline: diagnostic accuracy of CSF 14-3-3 protein in sporadic Creutzfeldt-Jakob disease: report of the guideline development subcommittee of the american academy of neurology. Neurology. 2012 Oct 2;79(14):1499–1506. PubMed PMID: 22993290; PubMed Central PMCID: PMC3525296.

 PubMed Web of Science ®Google Scholar

Satoh K, Shirabe S, Eguchi H, et al. 14-3-3 protein, total tau and phosphorylated tau in cerebrospinal fluid of patients with Creutzfeldt-Jakob disease and neurodegenerative disease in Japan. Cell Mol Neurobiol. 2006 Feb;26(1):45–52. PubMed PMID: 16633900.

 PubMed Web of Science ®Google Scholar

Aksamit AJ Jr., Preissner CM, Homburger HA. Quantitation of 14-3-3 and neuron-specific enolase proteins in CSF in Creutzfeldt-Jakob disease. Neurology. 2001 Aug 28;57(4):728–730. PubMed PMID: 11524493.

 PubMed Web of Science ®Google Scholar

Kapaki E, Kilidireas K, Paraskevas G, et al. Highly increased CSF tau protein and decreased β-amyloid (1–42) in sporadic CJD: a discrimination from Alzheimer’s disease? J Neurol Neurosurg. 2001;71(3):401–403.

 PubMed Web of Science ®Google Scholar

Llorens F, Kruse N, Karch A, et al. Validation of alpha-Synuclein as a csf biomarker for sporadic Creutzfeldt-Jakob Disease. Mol Neurobiol. 2018 Mar;55(3):2249–2257. 10.1007/s12035-017-0479-5. PubMed PMID: 28321768; PubMed Central PMCID: PMC5840235.

 PubMed Web of Science ®Google Scholar

Llorens F, Kruse N, Schmitz M, et al. Evaluation of alpha-synuclein as a novel cerebrospinal fluid biomarker in different forms of prion diseases. Alzheimers Dement. 2017 Jun;13(6):710–719. PubMed PMID: 27870938.

 PubMed Web of Science ®Google Scholar