2,146
Views
0
CrossRef citations to date
0
Altmetric
Research Paper

Machine learning model for predicting age in healthy individuals using age-related gut microbes and urine metabolites

, , , , , , , , , , , , & ORCID Icon show all
Article: 2226915 | Received 26 Oct 2022, Accepted 14 Jun 2023, Published online: 23 Jun 2023

ABSTRACT

Age-related gut microbes and urine metabolites were investigated in 568 healthy individuals using metataxonomics and metabolomics. The richness and evenness of the fecal microbiota significantly increased with age, and the abundance of 16 genera differed between the young and old groups. Additionally, 17 urine metabolites contributed to the differences between the young and old groups. Among the microbes that differed by age, Bacteroides and Prevotella 9 were confirmed to be correlated with some urine metabolites. The machine learning algorithm eXtreme gradient boosting (XGBoost) was shown to produce the best performing age predictors, with a mean absolute error of 5.48 years. The accuracy of the model improved to 4.93 years with the inclusion of urine metabolite data. This study shows that the gut microbiota and urine metabolic profiles can be used to predict the age of healthy individuals with relatively good accuracy.

Introduction

Chronological age is a major risk factor for functional impairment, chronic disease, and mortality. The complex social and medical costs associated with biological aging are continually increasing and represent an ever-growing challenge.Citation1 In keeping with the unprecedented growth rate of the global aging population, there is a need for a better understanding of the biological aging process and determinants of healthy aging.Citation2 As aging is a complex trait influenced by individual genetic factors, diet, lifestyle, and environmental factors, it is very difficult to measure the degree of aging simply by age. However, developing tools to measure aging is an important challenge for predicting health. Therefore, several studies have been conducted to measure aging with various biological age predictors.Citation3–10

The human gut contains a vast and diverse microbial ecosystem that has co-evolved with humans and is essential for human health.Citation11 The risk or severity of numerous diseases and disorders in a host is associated with the gut microbiome, and accurate trait prediction based on microbiome characteristics is an important issue.Citation12 During the past decade, accumulating evidence suggests that the gut microbiota affects body weight, energy homeostasis, innate immune system, and aging.Citation13,Citation14 Epidemiological studies have shown that loss of diversity in the core microbiota is associated with frailty.Citation15 Urine is a valuable resource for the early detection of sensitive biomarkers because it can rapidly reflect changes in the body. Furthermore, as an analytical tool, urine has advantages over other biofluids, such as relatively less complex sample pretreatment, lower protein content, and sample complexity, including fewer intermolecular interactions.Citation16 Lastly, urine is a more convenient sample to collect than blood because it can be taken non-invasively.Citation17 Therefore, urine can provide fingerprints for personalized endogenous metabolite markers that can interpret age.Citation18 Metabolic profiling of biofluids (such as urine, plasma, and fecal water) can be statistically integrated with microbial profile data using multivariate computational modeling.Citation19

Machine learning can analyze large-scale data and combine them into predictions of disease risk, diagnosis, prognosis, and appropriate treatments.Citation20 Recently, studies have been conducted to predict host characteristics by analyzing microbial and metabolic patterns using machine learning techniques.Citation21,Citation22 Predicting age using machine learning on multi-omics data can lead to better predictive models owing to interactions between features. A previous machine learning predictive model, a human microbiome clock, achieved an accuracy of 5.91 years mean absolute error in cross-validation.Citation23 However, biological differences, such as race, country, culture, and lifestyle, can lead to prediction errors.Citation24 Therefore, for industrial application, a population-based research design with biological characteristics similar to those of actual users is required.

In this study, age-related gut microbes and urine metabolites were identified in 568 healthy Koreans aged 20–80 years using a metataxonomics and metabolomics approach. In addition, urine metabolites associated with aging-related gut microbes were identified and an optimal age prediction model was built using machine learning technology.

Results and discussion

Gut microbial profiling by age

We performed 16S rRNA gene amplicon sequencing of 568 healthy individuals to investigate the pattern of gut microbes according to age. We divided the samples into three age groups based on their age distribution (young: 20–39 years, middle-aged: 40–59 years, and old: ≥60 years). The average relative abundances of Firmicutes and Bacteroidetes accounted for 89.29%, 90.40%, and 90.05% in the young, middle-aged, and old groups, respectively (). Recently, Leite et al.Citation25 reported a positive relationship of the small intestinal abundance of phylum Proteobacteria with age. To validate whether the phylum Proteobacteria is an indicator taxon of the aging process, we categorized our cohort into young, middle, and old groups and surveyed the relative abundance of the phylum Proteobacteria. Although a significant difference in the relative abundance between the groups was observed, the phylum Proteobacteria did not increase with age in Korean subjects (Supplementary Figure S1). Given that an increased prevalence of the intestinal Proteobacteria is a potential diagnostic signature of dysbiosis and risk of disease,Citation26 the irrelevance of Proteobacteria with age observed in this study might be attributed by i) our cohort comprising the specific disease-free healthy individuals and/or ii) compartmental difference (i.e., stool and small intestinal content) of microbial composition and structure.

Figure 1. Analysis of gut microbial community profiles according to age. The samples were divided into three groups (young: 20–39 years, middle-aged: 40–59 years, and old: ≥60 years). (a) Relative abundance (%) of bacteria by ascending age groups at the phylum level. (b) Principal component analysis (PCA) of beta-diversity based on the operational taxonomic unit (OTU) level (unweighted UniFrac). (c) Alpha-diversities of microbial communities between young (yellow), middle-aged (blue), and old (red) groups (p value was calculated using a Kruskal–Wallis test; *p < .1, **p < .01, ***p < .001). (d, e) Violin plots of relative abundance of bacterial taxa (genus level) that contribute to differences in the linear discriminant analysis effect size (LEfSe) (LDA >2.0 and p < .05) between young and old groups. (d) Relatively higher abundance in the young group. (e) Relatively higher abundance in the old group. p value was calculated using a Kruskal–Wallis test; *p < .1, **p < .01, ***p < .001.

Figure 1. Analysis of gut microbial community profiles according to age. The samples were divided into three groups (young: 20–39 years, middle-aged: 40–59 years, and old: ≥60 years). (a) Relative abundance (%) of bacteria by ascending age groups at the phylum level. (b) Principal component analysis (PCA) of beta-diversity based on the operational taxonomic unit (OTU) level (unweighted UniFrac). (c) Alpha-diversities of microbial communities between young (yellow), middle-aged (blue), and old (red) groups (p value was calculated using a Kruskal–Wallis test; *p < .1, **p < .01, ***p < .001). (d, e) Violin plots of relative abundance of bacterial taxa (genus level) that contribute to differences in the linear discriminant analysis effect size (LEfSe) (LDA >2.0 and p < .05) between young and old groups. (d) Relatively higher abundance in the young group. (e) Relatively higher abundance in the old group. p value was calculated using a Kruskal–Wallis test; *p < .1, **p < .01, ***p < .001.

Beta-diversity analysis based on the unweighted UniFrac index revealed that the distance between the young and middle-aged groups was smaller than the distance between the young and old groups (). However, the variance (PC1 = 16.5% and PC2 = 4.9%) in the principal component analysis was not high, indicating that the differences in microbial profiles between the groups were not significant. Alpha-diversity analysis using Chao1 showed considerable differences between the young and middle-aged groups and between the young and old groups (). In addition, the observed species and PD whole tree index results showed a pattern similar to that of Chao1. The Shannon index showed a significant difference only between the young and middle-aged groups. Although microbial diversity is generally known to decrease with age, conflicting results have been reported. Several studies have reported that microbial diversity continues to increase with age.Citation27,Citation28 For example, Yatsunenko et al.Citation24 reported that gut bacterial diversity increases with age. Kong et al.Citation29 showed that the gut microbial diversity increased in the longevity group (≥90 years old). The correlation between aging and microbial diversity may result from aging diseases.Citation29 In other words, as many diseases increase with age,Citation30,Citation31 disease rather than age may be the direct cause of the decrease in microbial diversity. In the present study of healthy individuals, the microbial diversity increased with age. Therefore, to establish a relationship between age and microbial diversity, the health status of the participants should be considered.

Linear discriminant analysis (LDA) effect size (LEfSe) analysis revealed 28 taxa that differed between the young and old groups (LDA >2.0) (Supplementary Figure S2). shows violin plots showing the differences in the gut microbes of the young, middle-aged, and old groups at the genus level (LDA >2.0, p < .05). When the taxonomic composition was compared between the young and old groups, the abundance of Clostridium sensu stricto 1, Holdemanella, Catenibacterium, Phascolarctobacterium, and Prevotella 9 was significantly higher in the old group than in the young group. In contrast, the abundance of Bacteroides, Barnesiella, Odoribacter, Collinsella, Bifidobacterium, Lachnoclostridium, Oscillibacter, Ruminiclostridium 5, UBA1819 (Family Ruminococcaceae), and Bilophila was lower in the old group than in the young group. In addition to age, sex influences the composition of the gut microbiome.Citation32 The results of the age-related gut microbial profile analysis in males and females are shown in Supplementary Figures S3 and S4, respectively. In the age group comparison, although the relative abundance of some microbes differed between males and females, most microbes showed similar patterns regardless of sex.

In this study, the young group had a higher relative abundance of Bacteroides but a lower relative abundance of Prevotella 9. The role of Bacteroides in aging has been inconsistent in literature. Wilmanski et al.Citation33 reported that a decrease in Bacteroides is a key feature of healthy aging. Conversely, Claesson et al.Citation34 reported an increased predominance of Bacteroides in the elderly group (≥65 years) compared to that in the young group. Similarly, the exact biological trends of Prevotella 9 associated with aging have not yet been established. According to Wei et al.,Citation35 the relative abundance of Prevotella 9 decreases with age, whereas the present study showed the opposite result. The contradictory results regarding the association of Prevotella 9 or Bacteroides with aging may be attributed to differences in dietary culture according to age. Gut microbial communities of individuals from developing countries were reported to be dominated by Prevotella,Citation36 whereas those from developed countries were highly abundant in Bacteroides.Citation24 Recently, the Korean diet has rapidly become westernized.Citation37 Specifically, young people prefer animal-derived foods, whereas middle-aged and older people prefer grain- and plant-based foods (Supplementary Table S1). Therefore, more precise studies are needed to verify the correlation between age-specific eating habits of Koreans and Bacteroides or Prevotella 9.

Our findings confirmed a high prevalence of Bifidobacterium in the young group. Bifidobacterium have been identified as almost ubiquitous inhabitants of the human host, playing an important role in the gut from birth.Citation38 Bifidobacterium plays an important role in the barrier effect and stimulation of the immune system associated with a range of beneficial health effects.Citation39 Several studies using different techniques have repeatedly observed a trend toward decreasing bifidobacteria in the elderly population. Although this decline was associated with a reduction in adhesion to the intestinal mucosa, whether this is due to changes in the microbiota or in the mucus structure remains unclear.Citation40 In addition, the extended use of antibiotics in the older population undoubtedly has a huge impact on the gut microbiota composition, decreasing the bifidobacterial population.Citation41

Metabolic profiling of urine by age

Metabolic differences in urine metabolites according to age were investigated using orthogonal partial least squares discriminant analysis (OPLS-DA). In the score plot, the young and old groups showed a tendency to separate, but the middle-aged group was distributed between the young and old groups (). Next, OPLS-DA was performed to identify metabolites that differed only between the young and old groups. The OPLS-DA score plot showed that the young and old groups were separated (). The metabolites responsible for discrimination are shown in (VIP >1.0, p < .05). Urine samples from the young group had higher levels of ribose, lactic acid, leucine, isoleucine, malonic acid, serine, fructose, threonine, glycine, and alanine and lower levels of glycolic acid, N-acetyl-D-glucosamine, serotonin, 3-methoxy-4-hydroxymandelate, and 2-oxobutyrate. The levels of mannitol and hippurate were the highest in the middle-aged group. To investigate sex-specific characteristics, the results of age-related metabolite analysis in males and females are shown in Supplementary Figures S5 and S6, respectively. In the age group comparison, although the content of some metabolites differed between males and females, most metabolites showed similar patterns regardless of sex.

Figure 2. Orthogonal partial least squares discriminant analysis (OPLS-DA) score plot obtained from gas chromatography mass spectrometry (GC-MS) data of urine. The samples were divided into three groups (young: 20–39 years, middle-aged: 40–59 years, and old: ≥60 years). (a) OPLS-DA score plot of young (yellow), middle-aged (blue), and old (red) groups. (b) OPLS-DA score plot of young (yellow) and old (red) groups. Cross validation was performed using a permutation test that was repeated 200 times. No over-fitting was observed. (c-e) Violin plots of identified metabolites that contribute to differentiation between the young and old groups in the OPLS-DA model (VIP >1.0 and p < .05). (c) Relatively higher intensity in the young group. (d) Relatively higher intensity in the middle-aged group. (e) Relatively higher intensity in the old group. p value was calculated using a Kruskal–Wallis test; *p < .1, **p < .01, ***p < .001.

Figure 2. Orthogonal partial least squares discriminant analysis (OPLS-DA) score plot obtained from gas chromatography mass spectrometry (GC-MS) data of urine. The samples were divided into three groups (young: 20–39 years, middle-aged: 40–59 years, and old: ≥60 years). (a) OPLS-DA score plot of young (yellow), middle-aged (blue), and old (red) groups. (b) OPLS-DA score plot of young (yellow) and old (red) groups. Cross validation was performed using a permutation test that was repeated 200 times. No over-fitting was observed. (c-e) Violin plots of identified metabolites that contribute to differentiation between the young and old groups in the OPLS-DA model (VIP >1.0 and p < .05). (c) Relatively higher intensity in the young group. (d) Relatively higher intensity in the middle-aged group. (e) Relatively higher intensity in the old group. p value was calculated using a Kruskal–Wallis test; *p < .1, **p < .01, ***p < .001.

A consistent underlying index of aging is a decline in cellular levels of the tripeptide glutathione.Citation42 Glutathione is an essential thiol antioxidant produced in the cytosol of all cells that plays a key role in protection against oxidative stress by neutralizing free radicals and reactive oxygen species.Citation42 The precursors of glutathione include cysteine, glutamate, and glycine. In the present study, glycine levels decreased with increasing age. The association between glycine and age may also be associated with the mitochondria. Hashizume et al.Citation43 reported that defects in glycine metabolism in mitochondria are in part responsible for the reduction in mitochondrial translation, and consequently lead to age-related respiratory defects. They also confirmed that glycine supplementation restored respiratory defects in elderly fibroblasts.Citation43 Serine metabolism is involved in cellular antioxidant functions. Serine supplementation increases Sirt1 protein deacetylase expression, decreases reactive oxygen species levels in the hypothalamus, and decreases pro-inflammatory cytokine levels.Citation44 Duan et al.Citation45 reported that impairment of glycine, serine, and threonine metabolism induces aging.

Hippurate is a glycine conjugate of benzoic acid that is associated with the gut microbiomeCitation46 and kidney function.Citation47 Previous animal and human studies have reported an association between hippurate use and age. In a study by Williams et al.,Citation48 a relatively low and variable amount of hippurate was observed in the urine of 4- and 6-week-old animals, but the levels increased and were less variable with increasing age. They argued that hippurate-like directional changes in urine are associated with maturation of the gut microbiota and the development of kidney function with age.Citation48 An age-dependent increase in urine hippurate concentration was also confirmed in an aging study in dogs.Citation49 Likewise, in the studies by Siqueira et al.Citation50 and Wijeyesekera et al.,Citation51 age and hippurate concentration were positively correlated. However, a study by Psihogios et al.Citation52 showed lower concentrations of hippurate in participants aged >50 years than in those aged <35 years. In the present study, the hippurate levels in the middle-aged group were higher than those in the young group, but the hippurate level of the old group was lower than that of the middle-aged group. As the levels of hippurate are significantly affected by diet, disease, and exposure to certain toxins,Citation46 a definitive interpretation is difficult. However, since many existing studies focus on urine hippurate as one of the biomarkers of aging, a research design that considers various variables such as race, disease, and diet is necessary for a clear interpretation.

Integration of urine metabolites and fecal microbes

Graphical outputs from data integration analysis for biomarker discovery using the latent components (DIABLO) showed significant positive or negative correlations between urine metabolites and fecal microbes (). Among the microbes that differed by age, Bacteroides and Prevotella 9 were confirmed to be correlated with some urine metabolites. shows the network analysis results derived from the circos plot. The relative abundance of Bacteroides, with high levels in the young group, was positively correlated with the levels of serine, leucine, isoleucine, and N-acetyl-D-glucosamine. Conversely, the relative abundance of Prevotella 9, which was high in the old group, was negatively correlated with the levels of threonine, serine, leucine, isoleucine, ribose, and N-acetyl-D-glucosamine.

Figure 3. Data integration analysis for biomarker discovery using latent components (DIABLO) graphical outputs on the Korean age study. (a) Circos plot of correlations between urine metabolites and fecal microorganisms. Each dataset is assigned a different color: metabolites are in blue blocks and microorganisms in green blocks. Red and blue lines indicate positive and negative correlations between two variables, respectively (r ≥ |0.50|). (b) Network visualization of urine metabolites and fecal microorganisms derived from circos plots. Each dataset is assigned a different color: metabolites are in blue circles and microorganisms in green circles. Red and green lines indicate positive and negative correlations between two variables, respectively.

Figure 3. Data integration analysis for biomarker discovery using latent components (DIABLO) graphical outputs on the Korean age study. (a) Circos plot of correlations between urine metabolites and fecal microorganisms. Each dataset is assigned a different color: metabolites are in blue blocks and microorganisms in green blocks. Red and blue lines indicate positive and negative correlations between two variables, respectively (r ≥ |0.50|). (b) Network visualization of urine metabolites and fecal microorganisms derived from circos plots. Each dataset is assigned a different color: metabolites are in blue circles and microorganisms in green circles. Red and green lines indicate positive and negative correlations between two variables, respectively.

Several studies have investigated the relationship between the gut microbiota and urine metabolites in diseases such as diabetes,Citation53 cardiovascular disease,Citation54 and depression.Citation55 Some metabolites produced by gut bacteria can enter the host bloodstream through the intestinal circulation.Citation56 Wikoff et al.Citation57 reported that one-third of the small molecules in the bloodstream may originate from gut bacteria. In the present study, Prevotella 9 and Bacteroides showed negative and positive correlations, respectively, with urine metabolites. Both microorganisms are related to human eating habits. However, as there are few studies on this aspect and further studies are needed to establish a clear association between gut microbes and urine metabolites.

Age prediction using machine learning

For machine learning modeling, 379 urine metabolite features and 269 fecal gut microbiota features were used. We applied four machine learning algorithms to this dataset: gradient boosting (GB), eXtreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and random forest (RF), to determine the optimal age prediction model. Supplementary Table S2 shows the overall mean absolute error of the different machine learning models. The machine learning algorithm XGBoost was shown to produce the best performing age predictors, with mean absolute error of 5.48 years. The accuracy of the model improved to 4.93 years with the inclusion of urine metabolite data. shows the results of a linear regression scatter plot between the age predicted by the XGBoost model and actual age (R2 = 0.79, p < .001). Various feature importance metrics can shed light on the overall importance of features. Based on the XGBoost model, each variable was ranked according to feature importance (Supplementary Table S3).

Figure 4. Linear regression scatter plot (Pearson correlation) of predicted age versus actual age based on gut microorganisms and urine metabolites (R2 = 0.79, p < .001). The x-axis shows the predicted age of the volunteers in years. The y-axis shows the actual age of the volunteers in years. Every blue dot displays one sample. The dotted line shows the linear correlation. R2 = coefficient of determination, MAE = mean absolute error in years.

Figure 4. Linear regression scatter plot (Pearson correlation) of predicted age versus actual age based on gut microorganisms and urine metabolites (R2 = 0.79, p < .001). The x-axis shows the predicted age of the volunteers in years. The y-axis shows the actual age of the volunteers in years. Every blue dot displays one sample. The dotted line shows the linear correlation. R2 = coefficient of determination, MAE = mean absolute error in years.

Using machine learning on multi-omics data to predict age can uncover new predictive models and biomarkers owing to interactions between features. This could be because of cross-interactions that occur when the value of one variable depends on another variable. In this case, even if a single variable does not seem to be significant for aging, the interaction term can contribute to predicting age. In this study, because the interaction of Actinobacteria and Bacteroidetes in machine learning contributed the most to improving the model performance, it is likely that it is the most important variable in age prediction. Another advantage is that machine learning methods are suitable for identifying biomarkers in variables with linear relationships. Microbiome and metabolite analyses require clear grouping for biomarker discovery. This study established three groups (young, middle-aged, and old) to identify age-related biomarkers and performed microbial community and metabolite analyses. However, this method is not suitable for variables with linear relationships. For example, the age variable was set according to the subjective opinion of the researcher in the group setting, because there was no clear distinguishing indicator. In this case, close variables (e.g., 39 years in the young group and 40 years in the middle-aged group) could cause errors in the potential biomarker search. In contrast, machine learning recognizes each linear variable as a separate variable and enables a biomarker search.

The aim of this study was to identify gut microbes and urine metabolites related to aging and to build an optimal age prediction model using machine learning technology. Age-related gut microbes and urine metabolites were identified in 568 healthy individuals. Although there is no minimum number of samples required to build a machine learning model, larger sample sizes can lead to the development of more robust models. Therefore, the use of more data, including open data, can improve the model performance. However, to use open data, it is necessary to match the sample analysis conditions and data-processing methods. Another limitation of the study was the uneven sample group size and female-to-male ratio. Unequal sample sizes between groups may affect the results of the statistical analysis. In addition, because the relative proportions of some microbes or metabolites may differ between sexes with age, obtaining data from a similar female-to-male ratio is important for building a robust model. Although 15 microbial genera and 17 metabolites showed differences between the young and old groups, some results differed from those of previous studies. This may be due to the different patterns of changes in gut microbes or urine metabolites with aging due to biological differences such as race, country, culture, and lifestyle. Nevertheless, this study showed that the gut microbiota and urine metabolic profiles can be used to predict the age of healthy individuals with relatively good accuracy, suggesting that noninvasive samples such as urine and fecal can provide aging-related information.

Materials and methods

Sample collection

Stool and urine samples were collected from volunteers at the Naju Korean Medicine Hospital, Dongshin University (n = 568). The major exclusion criteria were as follows: (1) regular intake of medications, prebiotics or probiotics, and antibiotics within the past 3 months; (2) intestinal-related diseases; (3) other pathologies (type I or type II diabetes, cardiovascular or cerebrovascular diseases, cancer, neurodegenerative disease, rheumatoid arthritis, and allergies); and (4) pregnancy and lactation.Citation58 The subjects fasted for 12 h before sample collection. Based on a previous study by Galkin et al.,Citation23 we analyzed the samples by dividing them into three age groups according to age distribution (young: 20–39 years, middle-aged: 40–59 years, and old: ≥60 years) (). General characteristics and anthropometric measurements of each age group are presented in Supplementary Table S4. Urine samples were collected on ice and stored at − 80°C until further analysis. Stool was collected using a stool sample collection kit (Fecal swab PLUS; Noble Biosciences Inc., Hwaseong, Korea). All stool samples were collected at the participants’ homes and immediately frozen (−20°C) in a freezer. It was then delivered to the laboratory within 24 h and stored in a − 80°C deep freezer. All samples were thawed in an ice bath before analysis. The study was conducted in accordance with relevant guidelines and regulations and the principles for human research. All the participants provided written informed consent. The medical ethics committee of Naju Korean Medicine Hospital of Dongshin University approved the study protocol (NJ-IRB-013).

Figure 5. Sample characteristics and workflow of young, middle-aged, and old groups.

Figure 5. Sample characteristics and workflow of young, middle-aged, and old groups.

Microbiota analysis

DNA extraction and 16S rRNA gene sequencing

Fecal DNA was extracted using the AccuFAST automation system (AccuGene Inc., Incheon, Korea), according to the manufacturer’s instructions. For MiSeq sequencing, bacterial genomic DNA amplification was performed using primers containing 515fb and 806rb and Nextera adaptor sequences to target the V4 hypervariable region of the 16S rRNA genes.Citation59 The 16S rRNA genes were amplified using 25 polymerase chain reaction (PCR) cycles and the KAPA HiFi HotStart ReadyMix (Roche Sequencing, Pleasanton, CA, USA). The PCR products (~250 bp) were purified using HiAccuBeads (AccuGene Inc., Incheon, Korea). Amplicon libraries were pooled at equimolar ratios. The pooled libraries were sequenced on an Illumina MiSeq system using the MiSeq Reagent Kit v2 for 500 cycles (Illumina, San Diego, CA, USA). The raw sequencing reads for all raw datasets were subjected to reference-based chimera filtering using VSEARCH v2.10.3.Citation60 Chimeric filtered sequences were assigned to operational taxonomic units (OTUs) by OTU picking using the QIIME pipeline (http://www.qiime.org). Sequences were clustered using UCLUST into OTUs based on the SILVA 132 database (pre-clustered at 97% similarity threshold).

Data processing

To estimate the alpha diversity (within samples), we calculated the Chao1, observed species, PD whole tree, Shannon, and Simpson indices. Beta diversity (between samples) was computed using unweighted UniFrac to produce distance matrices, which were subsequently used for grouping samples into hierarchical cluster trees with the unweighted pair group method with arithmetic mean (UPGMA) and principal coordinate analysis representations (PCoA). LEfSe (http://huttenhower.sph.harvard.edu/galaxy/) analysis was performed using a Galaxy computational tool to estimate the effect size of each differentially abundant feature,Citation61 with a cutoff value of the absolute log 10 LDA score > 2.0.

Metabolic analysis

Sample pre-processing

Urine was aliquoted into 200 µL samples and prepared, including the depletion of urea with urease (40 units). Urine metabolites were extracted by adding 1 mL cold methanol (70% v/v) to 30 μL urine. The samples were vortexed and incubated at 37°C for 30 min, followed by centrifugation at 12,000 rpm for 5 min at 4°C to remove precipitated proteins. Quality control samples were prepared by pooling equal volumes (approximately 10 μL) of each sample before derivatization. Ten microliters of ribitol (0.5 mg/mL) was added as the internal standard. Finally, the collected supernatant from each sample was concentrated and dehydrated using an Eppendorf vacuum centrifuge for 3 h at 45°C.

Sample derivatization

After drying, 100 μL O-methoxyamine hydrochloride in pyridine solution (20 mg/mL) was added to each sample. After vortexing each sample for 30 s, all samples were incubated at 30°C for 90 min in the dark. Silylation was performed by adding 50 μL N-methyl-N-trimethylsilyl-trifluoroacetamide containing 1% trimethylchlorosilane. After vortexing each sample for 30 s, the samples were incubated at 37°C for 30 min. After centrifuging the samples at 13,000 rpm for 10 min, the supernatants were subjected to Gas chromatography – mass spectrometry (GC-MS) analysis. To measure the performance and stability of the system together with the reproducibility of the sample treatment procedure, quality control samples were analyzed every 30 samples throughout the run.

GC-MS analysis

The derivatized samples were analyzed using GC-MS (QP2020, Shimadzu, Kyoto, Japan). Rtx-5 MS with a fused silica capillary column (30 m × 0.25 mm ID, J&W Scientific, Folsom, CA, USA) was used for the separation of metabolites. The front inlet temperature was set to 230°C. The column temperature was isothermally held at 80°C for 2 min, increased by 15°C/min to 330°C, and isothermally held for 6 min. The transfer line and ion source temperatures were 250°C and 200°C, respectively. Ionization was achieved with a 70 eV electron beam. The flow rate of helium gas through the column was 1 mL/min. Twenty scans per second were recorded over the mass range of 85–500 m/z. Chromatograms and mass spectra were acquired using Shimadzu GC solution (Shimadzu, Kyoto, Japan).

Data processing

GC-MS data were converted from Shimadzu GC-MS Postrun Analysis software (Shimadzu, Kyoto, Japan) to the netCDF format file and processed using MetAlign software for peak detection and alignment.Citation62 The resulting data (CSV-format file) were imported into AIoutput software for peak identification and prediction.Citation63 Data normalization was performed using the internal standard and total ion current values. OPLS-DA of GC-MS data was performed to visualize the variance of metabolites using SIMCA-P 17.0 (Umetrics, Umea, Sweden). Cross-validation was performed using a permutation test repeated 200 times. Metabolites with VIP > 1.0 and p < 0.05 were considered metabolites that could discriminate groups. Metabolites were identified by comparing their mass spectra with those in AIoutput software, NIST 20.0 library, standard reagents (Supplementary Figure S8), and the human metabolome database (HMDB, http://www.hmdb.ca).

Statistical analysis

Statistical analyses were performed using GraphPad Prism 9 (GraphPad Software, La Jolla, CA, USA). We evaluated the normality of the data using the D’Agostino-Pearson omnibus normality test. For parametric data, we conducted a Brown-Forsythe test to confirm equal variance, followed by either a one-way analysis of variance (ANOVA) or the Kruskal-Wallis test. To address multiple comparisons, we utilized the two-stage step-up method of Benjamini, Krieger, and Yekutieli. For non-parametric data, we used the Kruskal-Wallis test to compare groups, and for multiple comparisons, we employed Dunn’s post hoc test. We considered p values less than 0.05 to be statistically significant.

Integration between microbes and metabolites

The R package mixOmics was employed for the integration of the two datasets (fecal taxonomic profile and urine metabolite data) using the DIABLO model.Citation64

Machine learning analysis

Feature extraction

AutoEncoder was employed to learn additional information from nonlinear combinations of high-dimensional features by training a compressed representation of the raw features. AutoEncoder comprises an encoder and a decoder, where the encoder transforms the input data into a compressed representation, and the decoder reconstructs the representation back to the input data.Citation65 The encoder consists of four fully connected layers that contract 325 dimensions (as of the combined dataset) into 256, 128, 64, and 8 dimensional encoded vectors sequentially, whereas the decoder is a set of four fully connected layers that expand the encoded vectors into 64, 128, 256, and 325 dimensions. Each fully connected layer was followed by batch normalization,Citation66 and ReLU and dropoutCitation67 layers.

Feature selection

Permutation feature importance scores feature importance by permuting a feature and tracking the effect of shuffling on overall accuracy.Citation68 A total of 175 features (for the combined dataset) were utilized by specifying a subset of relevant features that exceeded the threshold using permutation feature importance scores.

Machine learning

Four different tree-based machine learning models: GB, XGBoost, LightGBM, and RF were applied to create age prediction models using metabolite and microbiome data.

Supplemental material

Supplemental Material

Download MS Word (4.9 MB)

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data presented in this study are openly available in MetaboLights repository at www.ebi.ac.uk/metabolights/MTBLS6313.

Supplemental material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/19490976.2023.2226915

Additional information

Funding

This research was supported by the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (No. 2022R1A5A2029546), the Korea Innovation Foundation (INNOPOLIS) grant funded by the Korean government (Ministry of Science and ICT) through a science and technology project that opens the future of the region (grant number: 2021-DD-UP-0380), a Korea University Grant, and the Institute of Biomedical Science and Food Safety, CJ-Korea University Food Safety Hall at Korea University, Republic of Korea.

References

  • Rae MJ, Butler RN, Campisi J, Grey ADNJ D, Finch CE, Gough M, Martin GM, Vijg J, Perrott KM, Logan BJ. The demographic and biomedical case for late-life interventions in aging. Sci Transl Med. 2010;2(40):40cm21. doi:10.1126/scitranslmed.30008.
  • Jylhävä J, Pedersen NL, Hägg S. Biological age predictors. EBioMedicine. 2017;21:29–14. doi:10.1016/j.ebiom.2017.03.046.
  • Blackburn EH, Greider CW, Szostak JW. Telomeres and telomerase: the path from maize, Tetrahymena and yeast to human cancer and aging. Nat Med. 2006;12(10):1133–1138. doi:10.1038/nm1006-1133.
  • Horvath S. DNA methylation age of human tissues and cell types. Genome Biol. 2013;14(10):1–20. doi:10.1186/gb-2013-14-10-r115.
  • Peters MJ, Joehanes RJ, Pilling LC, Schurmann C, Conneely KN, Powell J, Reinmaa E, Sutphin GL, Zhernakova A, Schramm K, et al. The transcriptional landscape of age in human peripheral blood. Nat Commun. 2015;6(1):1–14. doi:10.1038/ncomms9570.
  • Krištić J, Vučković F, Menni C, Klarić L, Keser T, Beceheli I, Pučić-Baković M, Novokmet M, Mangino M, Thaqi K, et al. Glycans are a novel biomarker of chronological and biological ages. J Gerontol A Biol Sci Med Sci. 2014;69(7):779–789. doi:10.1093/gerona/glt190.
  • Menni C, Kiddle SJ, Mangino M, Vinuela A, Psatha M, Steves C, Sattlecker M, Buil A, Newhouse S, Nelson S, et al. Circulating proteomic signatures of chronological age. J Gerontol A Biol Sci Med Sci. 2015;70(7):809–816. doi:10.1093/gerona/glu121.
  • Menni C, Kastenmüller G, Petersen AK, Bell JT, Psatha M, Tsai P-C, Gieger C, Schulz H, Erte I, John S, et al. Metabolomic markers reveal novel pathways of ageing and early development in human populations. Int J Epidemiol. 2013;42(4):1111–1119. doi:10.1093/ije/dyt094.
  • Hertel J, Friedrich N, Wittfeld K, Pietzner M, Budde K, Van der Auwera S, Lohmann T, Teumer A, Völzke H, Nauck M, et al. Measuring biological age via metabonomics: the metabolic age score. J Proteome Res. 2016;15(2):400–410. doi:10.1021/acs.jproteome.5b00561.
  • Sebastiani P, Thyagarajan B, Sun F, Schupf N, Newman AB, Montano M, Perls TT. Biomarker signatures of aging. Aging Cell. 2017;16(2):329–338. doi:10.1111/acel.12557.
  • Bengmark S. Ecological control of the gastrointestinal tract. The role of probiotic flora. Gut. 1998;42:2–7. doi:10.1136/gut.42.1.2.
  • Rothschild D, Weissbrod O, Barkan E, Kurilshikov A, Korem T, Zeevi D, Costea PI, Godneva A, Kalka IN, Bar N, et al. Environment dominates over host genetics in shaping human gut microbiota. Nature. 2018;555:210–215. doi:10.1038/nature25973.
  • Thevaranjan N, Puchta A, Schulz C, Naidoo A, Szamosi JC, Verschoor CP, Loukov D, Schenck LP, Jury J, Foley KP, et al. Age-associated microbial dysbiosis promotes intestinal permeability, systemic inflammation, and macrophage dysfunction. Cell Host & Microbe. 2017;21(4):455–66e4. doi:10.1016/j.chom.2017.03.002.
  • Wu CS, Muthyala SDV, Klemashevich C, Ufondu AU, Menon R, Chen Z, Devaraj S, Jayaraman A, Sun Y. Age-dependent remodeling of gut microbiome and host serum metabolome in mice. Aging. 2021;13(5):6330–6345. doi:10.18632/aging.202525.
  • Jeffery IB, Lynch DB, O’Toole PW. Composition and temporal stability of the gut microbiota in older persons. ISME J. 2016;10(1):170–182. doi:10.1038/ismej.2015.88.
  • Zhang A, Sun H, Wang P, Han Y, Wang X. Recent and potential developments of biofluid analyses in metabolomics. J Proteomics. 2012;74(4):1079–1088. doi:10.1016/j.jprot.2011.10.027.
  • Sun Y, Guo Z, Liu X, Yang L, Jing Z, Cai M, Zheng Z, Shao C, Yefan Z, Sun H, et al. Noninvasive urinary protein signatures associated with colorectal cancer diagnosis and metastasis. Nat Commun. 2022;19(1):2757. doi:10.1038/s41467-022-30391-8.
  • Adav SS, Wang Y. Metabolomics signatures of aging: recent advances. Aging Dis. 2021;12(2):646–661. doi:10.14336/AD.2020.0909.
  • Nicholson JK, Holmes E, Kinross J, Burcelin R, Gibson G, Jia W, Pettersson S. Host-gut microbiota metabolic interactions. Science. 2012;336(6086):1262–1267. doi:10.1126/science.1223813.
  • Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347–1358. doi:10.1056/NEJMra1814259.
  • Knights D, Costello EK, Knight R. Supervised classification of human microbiota. FEMS Microbiol Rev. 2011;35(2):343–359. doi:10.1111/j.1574-6976.2010.00251.x.
  • Rist MJ, Roth A, Frommherz L, Weinert CH, Krüger R, Merz B, Bunzel D, Mack C, Egert B, Bub A, et al. Metabolite patterns predicting sex and age in participants of the Karlsruhe metabolomics and nutrition (KarMen) study. Plos One. 2017;12(8):e0183228. doi:10.1371/journal.pone.0183228.
  • Galkin F, Mamoshina P, Aliper A, Putin E, Moskalev V, Gladyshev VN, Zhavoronkov A. Human gut microbiome aging clock based on taxonomic profiling and deep learning. iScience. 2020;23(6):101199. doi:10.1016/j.isci.2020.101199.
  • Yatsunenko T, Rey FE, Manary MJ, Trehan I, Dominguez-Bello MG, Contreras M, Magris M, Hidalgo G, Baldassano RN, Anokhin AP, et al. Human gut microbiome viewed across age and geography. Nature. 2012;486(7402):222–227. doi:10.1038/nature11053.
  • Leite G, Pimentel M, Barlow GM, Chang C, Hosseini A, Wang J, Parodi G, Sedighi R, Rezaie A, Mathur R. Age and the aging process significantly alter the small bowel microbiome. Cell Rep. 2021;36(13):109765. doi:10.1016/j.celrep.2021.109765.
  • Shin NR, Whon TW, Bae JW. Proteobacteria: microbial signature of dysbiosis in gut microbiota. Trends Biotechonol. 2015;33(9):496–503. doi:10.1016/j.tibtech.2015.06.011.
  • Odamaki T, Kato K, Sugahara H, Hashikura N, Takahashi S, Xiao J, Abe F, Osawa R. Age-related changes in gut microbiota composition from newborn to centenarian: a cross-sectional study. BMC microbiol. 2016;16(1):1–12. doi:10.1186/s12866-016-0708-5.
  • Bian G, Gloor GB, Gong A, Jia C, Zhang W, Hu J, Zhang H, Zhang Y, Zhou Z, Zhang J, et al. The gut microbiota of healthy aged Chinese is similar to that of the healthy young. Msphere. 2017;2(5):e00327–17. doi:10.1128/mSphere.00327-17.
  • Kong F, Deng F, Li Y, Zhao J. Identification of gut microbiome signatures associated with longevity provides a promising modulation target for healthy aging. Gut Microbes. 2019;10(2):210–215. doi:10.1080/19490976.2018.1494102.
  • Turnbaugh PJ, Hamady M, Yatsunenko T, Cantarel BL, Duncan A, Ley RE, Sogin ML, Jones WJ, Roe BA, Affourtit JP, et al. A core gut microbiome in obese and lean twins. Nature. 2009;457(7228):480–484. doi:10.1038/nature07540.
  • Jackson MA, Jeffery IB, Beaumont M, Bell JT, Clark AG, Ley RE, O’Toole PW, Spector TD, Steves CJ. Signatures of early frailty in the gut microbiota. Genome Med. 2016;8(1):1–11. doi:10.1186/s13073-016-0262-7.
  • Zhernakova A, Kurilshikov A, Bonder MJ, Tigchelaar EF, Schirmer M, Vatanen T, Mujagic Z, Vila AV, Falony G, Vieira-Silva S, et al. Population-based metagenomics analysis reveals markers for gut microbiome composition and diversity. Science. 2016;352(6285):565–569. doi:10.1126/science.aad3369.
  • Wilmanski T, Diener C, Rappaport N, Patwardhan S, Wiedrick J, Lapidus J, Earls JC, Zimmer A, Glusman G, Robinson M, et al. Gut microbiome pattern reflects healthy ageing and predicts survival in humans. Nat Metab. 2021;3(2):274–286. doi:10.1038/s42255-021-00348-0.
  • Claesson MJ, Cusack S, O’Sullivan O, Greene-Diniz R, de Weerd H, Flannery E, Marchesi JR, Falush D, Dinan T, Fitzgerald G, et al. Composition, variability, and temporal stability of the intestinal microbiota of the elderly. Proc Natl Acad Sci USA. 2011;108(1):4586–4591. doi:10.1073/pnas.1000097107.
  • Wei Z-Y, Rao J-H, Tang M-T, Zhao G-A, Li Q-C, Wu L-M, Liu S-Q, Li B-H, Xiao B-Q, Liu X-Y, et al. Characterization of changes and driver microbes in gut microbiota during healthy aging using a captive monkey model. Genom, Proteom & Bioinform. 2021;S1672-0229(21):00258–8. doi:10.1016/j.gpb.2021.09.009.
  • Lin A, Bik EM, Costello EK, Dethlefsen L, Haque R, Relman DA, Singh U, Aziz RK. Distinct distal gut microbiome diversity and composition in healthy children from Bangladesh and the United States. Plos One. 2013;8(1):e53838. doi:10.1371/journal.pone.0053838.
  • Kim JH, Lee J, Choi IJ, Kim Y-I, Kim J. Dietary patterns and gastric cancer risk in a Korean population: a case–control study. Eur J Nutr. 2021;60(1):389–397. doi:10.1007/s00394-020-02253-w.
  • Mattarelli B. The family bifidobacteriaceae. New York: Springer; 2006.
  • Picard C, Fioramonti J, Francois A, Robinson T, Neant F, Matuchansky C. Review article: bifidobacteria as probiotic agents–physiological effects and clinical benefits. Aliment Pharmacol Ther. 2005;22(6):495–512. doi:10.1111/j.1365-2036.2005.02615.x.
  • Arboleya S, Watkins C, Stanton C, RPJFim R. Gut bifidobacteria populations in human health and aging. Front Microbiol. 2016;7:1204. doi:10.3389/fmicb.2016.01204.
  • O’sullivan O, Coakley M, Lakshminarayanan B, Conde S, Claesson MJ, Cusack S, Fitzgerald AP, O’Toole PW, Stanton C, Ross RP. Alterations in intestinal microbiota of elderly Irish subjects post-antibiotic therapy. J Antimicrob Chemother. 2013;68(1):214–221. doi:10.1093/jac/dks348.
  • Ferguson G, Bridge W. Glutamate cysteine ligase and the age-related decline in cellular glutathione: the therapeutic potential of γ-glutamylcysteine. Arch Biochem Biophys. 2016;593:12–23. doi:10.1016/j.abb.2016.01.017.
  • Hashizume O, Ohnishi S, Mito T, Shimizu A, Ishikawa K, Nakada K, Soda M, Mano H, Togayachi S, Miyoshi H, et al. Epigenetic regulation of the nuclear-coded GCAT and SHMT2 genes confers human age-associated mitochondrial respiration defects. Sci Rep. 2015;5(1):1–11. doi:10.1038/srep10434.
  • Zhou X, Zhang H, He L, Wu X, Yin Y. Long-term l-serine administration reduces food intake and improves oxidative stress and Sirt1/NFκB signaling in the hypothalamus of aging mice. Front Endocrinol. 2018;9:476. doi:10.3389/fendo.2018.00476.
  • Duan D, Wang K, Zhou Y, Qin X, Gao L, Du G. Baicalein exerts beneficial effects in d-galactose-induced aging rats through attenuation of inflammation and metabolic dysfunction. Rejuvenation Res. 2017;20(6):506–516. doi:10.1089/rej.2017.1919.
  • Lees HJ, Swann JR, Wilson ID, Nicholson JK, Holmes E. Hippurate: the natural history of a mammalian–microbial cometabolite. J Proteome Res. 2013;12(4):1527–1546. doi:10.1021/pr300900b.
  • Grams ME, Tin A, Rebholz CM, Shafi T, Köttgen A, Perrone RD, Sarnak MJ, Inker LA, Levey AS, Coresh J. Metabolomic alterations associated with cause of CKD. Clin J Am Soc Nephrol. 2017;12(11):1787–1794. doi:10.2215/CJN.02560317.
  • Williams RE, Lenz EM, Lowden JS, Rantalainen M, Wilson ID. The metabonomics of aging and development in the rat: an investigation into the effect of age on the profile of endogenous metabolites in the urine of male rats using 1H NMR and HPLC-TOF MS. Mol Biosyst. 2005;1(2):166–175. doi:10.1039/B500852B.
  • Wang Y, Lawler D, Larson B, Ramadan Z, Kochhar S, Holmes E, Nicholson JK. Metabonomic investigations of aging and caloric restriction in a life-long dog study. J Proteome Res. 2007;6(5):1846–1854. doi:10.1021/pr060685n.
  • Siqueira MEPB, Paiva MJN. Hippuric acid in urine: reference values. Rev Saude Publica. 2002;36:723–727. doi:10.1590/S0034-89102002000700010.
  • Wijeyesekera A, Clarke PA, Bictash M, Brown IJ, Fidock M, Ryckmans T, Yap IKS, Chan Q, Stamler J, Elliott P, et al. Quantitative UPLC-MS/MS analysis of the gut microbial co-metabolites phenylacetylglutamine, 4-cresyl sulphate and hippurate in human urine: INTERMAP Study. Anal Methods. 2012;4(1):65–72. doi:10.1039/C1AY05427A.
  • Psihogios NG, Gazi IF, Elisaf MS, Seferiadis KI, Bairaktari ET. Gender‐related and age‐related urinalysis of healthy subjects by NMR‐based metabonomics. NMR Biomed. 2008;21(3):195–207. doi:10.1002/nbm.1176.
  • Goytia M, Shafer WM. Polyamines can increase resistance of Neisseria gonorrhoeae to mediators of the innate human host defense. Infect Immun. 2010;78(7):3187–3195. doi:10.1128/IAI.01301-09.
  • Wang Z, Klipfell E, Bennett BJ, Koeth R, Levison BS, DuGar B, Feldstein AE, Britt EB, Fu X, Chung YM, et al. Gut flora metabolism of phosphatidylcholine promotes cardiovascular disease. Nature. 2011;472(7341):57–63. doi:10.1038/nature09922.
  • Zheng S, Yu M, Lu X, Huo T, Ge L, Yang J, Wu C, Li F. Urinary metabonomic study on biochemical changes in chronic unpredictable mild stress model of depression. Clin Chim Acta. 2010;411(3–4):204–209. doi:10.1016/j.cca.2009.11.003.
  • Zhang C, Zhao L. Strain-level dissection of the contribution of the gut microbiome to human metabolic disease. Genome Med. 2016;8(1):1–10. doi:10.1186/s13073-016-0304-1.
  • Wikoff WR, Anfora AT, Liu J, Schultz PG, Lesley SA, Peters EC, Siuzdak G. Metabolomics analysis reveals large effects of gut microflora on mammalian blood metabolites. PNAS. 2009;106(10):3698–3703. doi:10.1073/pnas.0812874106.
  • De Filippis F, Pellegrini N, Vannini L, Jeffery IB, La Storia A, Laghi L, Serrazanetti DI, Cagno RD, Ferrocino I, Lazzi C, et al. High-level adherence to a Mediterranean diet beneficially impacts the gut microbiota and associated metabolome. Gut. 2016;65(11):1812–1821. doi:10.1136/gutjnl-2015-309957.
  • Parada AE, Needham DM, Fuhrman JA. Every base matters: assessing small subunit rRNA primers for marine microbiomes with mock communities, time series and global field samples. Environ Microbiol. 2016;18(5):1403–1414. doi:10.1111/1462-2920.13023.
  • Rognes T, Flouri T, Nichols B, Quince C, Mahé F. VSEARCH: a versatile open source tool for metagenomics. PeerJ. 2016;4:e2584. doi:10.7717/peerj.2584.
  • Segata N, Izard J, Waldron L, Gevers D, Miropolsky L, Garrett WS, Huttenhower C. Metagenomic biomarker discovery and explanation. Genome Biol. 2011;12(6):1–18. doi:10.1186/gb-2011-12-6-r60.
  • Lommen A. MetAlign: interface-driven, versatile metabolomics tool for hyphenated full-scan mass spectrometry data preprocessing. Anal Chem. 2009;81(8):3079–3086. doi:10.1021/ac900036d.
  • Tsugawa H, Bamba T, Shinohara M, Nishiumi S, Yoshida M, Fukusaki E. Practical non-targeted gas chromatography/mass spectrometry-based metabolomics platform for metabolic phenotype analysis. J Biosci Bioeng. 2011;112(3):292–298. doi:10.1016/j.jbiosc.2011.05.001.
  • Singh A, Shannon CP, Gautier B, Rohart F, Vacher M, Tebbutt SJ, Cao K-A L, Birol I. DIABLO: an integrative approach for identifying key molecular drivers from multi-omics assays. Bioinformatics. 2019;35(17):3055–3062. doi:10.1093/bioinformatics/bty1054.
  • Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Leam Res. 2010;11:3371–3408.
  • Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. Int Conf Mach learn. Lille, France. 2015:448–456.
  • Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15:1929–1958.
  • Altmann A, Tolosi L, Sander O, Lengauer T. Permutation importance: a corrected feature importance measure. Bioinformatics. 2010;26:1340–1347. doi:10.1093/bioinformatics/btq134.