Machine learning-based prediction of proteins’ architecture using sequences of amino acids and structural alphabets

Figures & data

Figure 1. Schematic comparison between SCOP and CATH.

Figure 2. Two structures belong to the same Architecture (3-Layer (aba) Sandwich) with different Topologies (Left-to-right: (1) Rossmann fold, PDBID: 1A2B and (2) Ribosomal Protein L9, PDBID: 2HBA). Although the number and length of the secondary structures is quite different, they both share the same overall shape: beta-sheets surrounded by alpha helices. Images created using Mol* (Sehnal et al., 2021).

Figure 3. Distribution of 40 CATH Architectures amongst the 169,843 domains – Dataset #1.

Figure 4. Distribution of 12 CATH Architectures amongst the 157,044 domains – Dataset #2.

Figure 5. Distribution of 28 excluded CATH Architectures amongst the 12,799 domains – This represents Dataset #1 subtracted by Dataset #2.

Figure 6. Flowchart of the Process to create both Datasets: 1 & 2 that contain in each row (domain), the Amino Acid Sequence, Protein Block Sequence and the Architecture ID.

Figure 7. F1 Score of three algorithms at three experiments for both datasets. The first column is for Dataset #2 (12 classes) and the second column is for Dataset #1 (40 classes).

Table 1. Results (F1 Score) of Experiments 1 (using Sequence of Amino Acids), Experiments 2 (using Sequence of Protein Blocks) and Experiments 3 (Using Both Features) for the main Classes (Dataset 2 – 12 Architectures).

12 Classes – Experiments 1, 2 and 3
	F1 Score
	SVM			Logistic Regression			Random Forests
	AA- Based	PB- Based	Both	AA- Based	PB- Based	Both	AA- Based	PB- Based	Both
K = 2	0.54	0.43	0.61	0.57	0.48	0.66	0.91	0.88	0.92
K = 3	0.89	0.64	0.91	0.88	0.57	0.75	0.91	0.88	0.92
K = 4	0.94	0.78	0.95	0.93	0.62	0.77	0.91	0.88	0.90
K = 5	0.94	0.84	0.95	0.93	0.68	0.79	0.91	0.89	0.90
K = 6	0.94	0.87	0.95	0.92	0.72	0.82	0.91	0.90	0.90
K = 7	0.93	0.89	0.95	0.92	0.74	0.83	0.90	0.91	0.91
K = 8	0.93	0.91	0.95	0.92	0.75	0.84	0.90	0.91	0.91
K = 9	0.93	0.92	0.96	0.91	0.76	0.84	0.90	0.91	0.91
K = 10	0.92	0.92	0.96	0.91	0.77	0.85	0.90	0.91	0.91
K = 11	0.92	0.94	0.96	0.91	0.78	0.85	0.90	0.91	0.91
K = 12	0.92	0.94	0.96	0.91	0.78	0.86	0.90	0.91	0.91
K = 13	0.92	0.94	0.96	0.91	0.78	0.86	0.89	0.91	0.91
K = 14	0.91	0.94	0.96	0.91	0.79	0.87	0.89	0.91	0.91
K = 15	0.91	0.94	0.96	0.90	0.80	0.87	0.89	0.91	0.91
K = 16	0.91	0.94	0.95	0.90	0.80	0.88	0.89	0.90	0.91
K = 17		0.94	0.95		0.80	0.88		0.90	0.91
K = 18		0.92	0.95		0.81	0.89		0.89	0.91
K = 19		0.92	0.95		0.81	0.90		0.89	0.91
K = 20		0.92	0.95		0.81	0.90		0.89	0.91
K = 21		0.91	0.95		0.82	0.90		0.88	0.91
K = 22		0.91	0.94		0.81	0.91		0.88	0.90
K = 23		0.91	0.94		0.81	0.91		0.88	0.90
K = 24		0.90	0.94		0.81	0.91		0.87	0.90
K = 25		0.91	0.94		0.80	0.91		0.87	0.89
K = 26		0.90	0.93		0.79	0.91		0.86	0.89
K = 27		0.90	0.93		0.77	0.90		0.86	0.89
K = 28		0.89	0.93		0.77	0.90		0.85	0.88
K = 29		0.88	0.93		0.76	0.90		0.85	0.88
K = 30		0.88	0.92		0.75	0.90		0.84	0.88
K = 31		0.88	0.92		0.74	0.89		0.83	0.88
K = 32		0.87	0.92		0.72	0.89		0.82	0.87
AVG	0.90	0.88	0.93	0.89	0.75	0.86	0.90	0.88	0.90
MAX	0.94	0.94	0.96	0.93	0.82	0.91	0.91	0.91	0.92

Table 2. Results (F1 Score) of Experiments 1 (using Sequence of Amino Acids), Experiments 2 (using Sequence of Protein Blocks) and Experiments 3 (Using Both Features) for All Classes (Dataset 1–40 Architectures).

40 Classes – Experiments 1, 2 and 3
	F1 Score
	SVM			Logistic Regression			Random Forests
	AA- Based	PB- Based	Both	AA- Based	PB- Based	Both	AA- Based	PB- Based	Both
K = 2	0.71	0.51	0.75	0.67	0.23	0.40	0.89	0.83	0.89
K = 3	0.83	0.69	0.86	0.87	0.28	0.44	0.90	0.82	0.87
K = 4	0.92	0.77	0.93	0.90	0.33	0.47	0.90	0.83	0.85
K = 5	0.92	0.79	0.93	0.91	0.38	0.51	0.90	0.84	0.86
K = 6	0.92	0.81	0.93	0.91	0.43	0.55	0.90	0.85	0.86
K = 7	0.91	0.83	0.93	0.91	0.46	0.57	0.90	0.86	0.86
K = 8	0.91	0.86	0.93	0.91	0.47	0.59	0.89	0.86	0.87
K = 9	0.91	0.87	0.94	0.90	0.49	0.60	0.89	0.86	0.86
K = 10	0.90	0.88	0.94	0.90	0.51	0.61	0.89	0.86	0.87
K = 11	0.90	0.89	0.94	0.90	0.52	0.63	0.89	0.86	0.87
K = 12	0.90	0.90	0.94	0.90	0.53	0.66	0.89	0.86	0.87
K = 13	0.90	0.90	0.94	0.90	0.55	0.68	0.88	0.86	0.87
K = 14	0.90	0.90	0.94	0.90	0.58	0.70	0.88	0.86	0.87
K = 15	0.90	0.90	0.94	0.89	0.60	0.72	0.88	0.86	0.87
K = 16	0.89	0.90	0.93	0.89	0.62	0.75	0.88	0.86	0.87
K = 17		0.90	0.93		0.63	0.77		0.85	0.87
K = 18		0.90	0.93		0.65	0.79		0.85	0.87
K = 19		0.90	0.93		0.66	0.80		0.85	0.87
K = 20		0.90	0.93		0.67	0.81		0.85	0.87
K = 21		0.89	0.93		0.67	0.81		0.85	0.87
K = 22		0.89	0.92		0.66	0.82		0.84	0.88
K = 23		0.89	0.92		0.65	0.82		0.84	0.87
K = 24		0.88	0.92		0.65	0.82		0.82	0.87
K = 25		0.88	0.92		0.64	0.82		0.83	0.86
K = 26		0.87	0.92		0.63	0.82		0.82	0.87
K = 27		0.87	0.92		0.62	0.81		0.81	0.86
K = 28		0.86	0.91		0.61	0.81		0.81	0.85
K = 29		0.86	0.91		0.60	0.80		0.80	0.84
K = 30		0.85	0.91		0.59	0.79		0.79	0.84
K = 31		0.84	0.91		0.58	0.79		0.78	0.83
K = 32		0.84	0.91		0.57	0.78		0.77	0.83
AVG	0.89	0.85	0.92	0.88	0.55	0.70	0.89	0.84	0.86
MAX	0.92	0.90	0.94	0.91	0.67	0.82	0.90	0.86	0.89

Table 3. Comparison between averages of Table 1 with Table 2.

		SVM			Logistic Regression			Random Forests
		AA-Based	PB-Based	Both	AA-Based	PB-Based	Both	AA-Based	PB-Based	Both
Average of F1 Score	12 Classes	0.90	0.88	0.93	0.89	0.75	0.86	0.90	0.88	0.90
	40 Classes	0.89	0.85	0.92	0.88	0.55	0.70	0.89	0.84	0.86
	Difference	0.01	0.03	0.01	0.01	0.20	0.16	0.01	0.04	0.04
Highest F1 Score	12 Classes	0.94	0.94	0.96	0.93	0.82	0.91	0.91	0.91	0.92
	40 Classes	0.92	0.90	0.94	0.91	0.67	0.82	0.90	0.86	0.89
	Difference	0.02	0.04	0.02	0.02	0.15	0.09	0.01	0.05	0.03

Table 4. Results (F1 Score) of Extended Experiments 3 (using both features, Sequence of Amino Acids and Sequence of Protein Blocks) for both the main and 40 Classes (Datasets 1 & 2).

12 Classes – Extended Experiments 3
	F1 Score
	SVM	Logistic Regression	Random Forests
k= 5 (AA) K = 10 (PB)			0.92
k= 5 (AA) K = 13 (PB)	0.97
k= 5 (AA) K = 21 (PB)		0.91
40 Classes – Extended Experiments 3
	SVM	Logistic Regression	Random Forests
k= 5 (AA) K = 11 (PB)			0.87
k= 5 (AA) K = 13 (PB)	0.95
k= 5 (AA) K = 25 (PB)		0.83

Figure 8. From Left to Right, structures of three proteins that belong to architectures 6.10 and 6.20: 2LH0 (6.10), 4GIP (6.10), 2CON (6.20). Images created using Mol* (Sehnal et al., 2021).

Table 5. Detailed F1 Score for SVM – 12 Classes – k = 5 (AA) K = 13 (PB).

Architecture	Size	F1	Recall	Precision
1.1	29,458	0.97	0.99	0.96
1.2	9,880	0.96	0.95	0.97
1.25	1,828	0.95	0.91	1
2.3	3,684	0.95	0.91	0.99
2.4	12,283	0.98	0.97	0.99
2.6	18,101	0.98	0.99	0.98
3.1	7,182	0.96	0.94	0.99
3.2	7,625	0.98	0.97	1
3.3	29,638	0.97	0.98	0.96
3.4	38,245	0.98	0.99	0.98
3.5	1,870	0.98	0.96	1
3.9	10,871	0.97	0.95	1
Average	14,222	0.97	0.96	0.99
Accuracy	0.97

Table 6. Detailed F1 Score for SVM – 12 Classes – k = 5 (AA) K = 13 (PB).

Architecture	Size	F1	Recall	Precision
1.1	29,458	0.97	0.99	0.95
1.2	9,880	0.95	0.95	0.95
1.25	1,828	0.95	0.92	0.99
1.5	498	0.97	0.94	1
2.1	1,597	0.96	0.93	0.99
2.101	130	0.95	0.91	1
2.102	117	0.97	0.95	1
2.11	31	0.93	0.87	1
2.115	205	0.98	0.96	0.99
2.12	571	0.98	0.96	1
2.13	667	0.98	0.97	1
2.14	176	0.99	0.98	1
2.15	30	0.95	0.93	0.97
2.16	510	0.97	0.94	1
2.17	941	0.97	0.95	1
2.2	646	0.89	0.82	0.97
2.3	3,684	0.95	0.92	0.98
2.4	12,283	0.98	0.97	0.99
2.5	39	0.87	0.77	1
2.6	18,101	0.98	0.99	0.97
2.7	1,522	0.97	0.95	1
2.8	704	0.98	0.96	1
2.9	53	0.99	0.98	1
3.1	7,182	0.96	0.94	0.99
3.101	19	0.97	0.95	1
3.15	32	0.81	0.69	1
3.2	7,625	0.98	0.97	1
3.3	29,638	0.96	0.97	0.95
3.4	38,245	0.98	0.99	0.97
3.5	1,870	0.99	0.97	1
3.55	149	0.92	0.86	1
3.6	1,806	0.97	0.95	1
3.65	204	0.99	0.99	1
3.7	106	0.97	0.95	0.99
3.75	93	0.94	0.88	1
3.8	358	0.97	0.94	1
3.9	10,871	0.97	0.95	1
4.1	1,264	0.92	0.86	0.99
6.1	1,012	0.79	0.67	0.98
6.2	197	0.83	0.75	0.94
Average	4,608	0.95	0.92	0.99
Accuracy	0.97

Figure 9. F1 Scores of 40-Classes SVM 80 experiments for each Architecture.

Table 7. Holdout dataset’s results.

12 Classes – Experiments 1, 2 and 3
	F1 Score
	SVM			Logistic Regression			Random Forests
	AA-Based	PB-Based	Both	AA-Based	PB-Based	Both	AA-Based	PB-Based	Both
K = 2									0.93
K = 5	0.95			0.94			0.93
K = 10								0.91
K = 13		0.94	0.97
K = 21					0.83
K = 25						0.92
40 Classes – Experiments 1, 2 and 3
	F1 Score
	SVM			Logistic Regression			Random Forests
	AA-Based	PB-Based	Both	AA-Based	PB-Based	Both	AA-Based	PB-Based	Both
K = 2									0.89
K = 5	0.92			0.91			0.91
K = 10								0.85
K = 13		0.90	0.94
K = 21					0.69
K = 25						0.83
12 Classes – Extended Experiments 3
	F1 Score
	SVM	Logistic Regression	Random Forests
k= 5 (AA) K = 10 (PB)			0.92
k= 5 (AA) K = 13 (PB)	0.97
k= 5 (AA) K = 21 (PB)		0.91
40 Classes – Extended Experiments 3
	SVM	Logistic Regression	Random Forests
k= 5 (AA) K = 11 (PB)			0.88
k= 5 (AA) K = 13 (PB)	0.94
k= 5 (AA) K = 25 (PB)		0.83

Table 8. Final conclusion that includes the best 4 algorithms out of the study.

	Proteins/Domains of Known Structures		Proteins/Domains of Unknown Structures
	12 Architectures	40 Architectures	12 Architectures	40 Architectures
Solution	Algorithm: SVM Features: Sequences of AA and PB (K = 13)		Algorithm: SVM Features: Sequences of AA (K = 5)
Expected F1 Score	0.97	0.94	0.95	0.92

Sehnal, D., Bittrich, S., Deshpande, M., Svobodová, R., Berka, K., Bazgier, V., Velankar, S., Burley, S. K., Koča, J., & Rose, A. S. (2021). Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures. Nucleic Acids Research, 49(W1), W431–W437. https://doi.org/10.1093/nar/gkab314

 PubMed Web of Science ®Google Scholar

Data availability statement

The code and datasets developed in this study can be accessed at https://zenodo.org/records/10203431.