Peeling off the layers from microbial dark matter (MDM): recent advances, future challenges, and opportunities

Abstract

Microbes represent the most common organisms on Earth; however, less than 2% of microbial species in the environment can undergo cultivation for study under laboratory conditions, and the rest of the enigmatic, microbial world remains mysterious, constituting a kind of “microbial dark matter” (MDM). In the last two decades, remarkable progress has been made in culture-dependent and culture-independent techniques. More recently, studies of MDM have relied on culture-independent techniques to recover genetic material through either unicellular genomics or shotgun metagenomics to construct single-amplified genomes (SAGs) and metagenome-assembled genomes (MAGs), respectively, which provide information about evolution and metabolism. Despite the remarkable progress made in the past decades, the functional diversity of MDM still remains uncharacterized. This review comprehensively summarizes the recently developed culture-dependent and culture-independent techniques for characterizing MDM, discussing major challenges, opportunities, and potential applications. These activities contribute to expanding our knowledge of the microbial world and have implications for various fields including Biotechnology, Bioprospecting, Functional genomics, Medicine, Evolutionary and Planetary biology. Overall, this review aims to peel off the layers from MDM, shed light on recent advancements, identify future challenges, and illuminate the exciting opportunities that lie ahead in unraveling the secrets of this intriguing microbial realm.

Keywords:

• Microbiome
• metagenomics
• metagenome assembled genome
• single amplified genome
• artificial intelligence

Introduction

Microbiota inhabiting diverse environmental conditions are extensively studied, revealing numerous new genes and species (Rinke et al. 2013; Sunagawa et al. 2015; Thompson et al. 2017; Proctor et al. 2019; Murray et al. 2020). Despite this, many microbial species still need to be discovered and described, making up a cesspool of unexplained microbes called microbial dark matter (MDM). As microbial communities live in diverse habitats, therefore, several fields are expected to be improved with the advances in MDM research (Shenhav et al. 2019; Mitchell et al. 2020): Microbial ecology; Microbes found in general habitats like freshwaters (Tran et al. 2021), soils (Yi et al. 2021), and gut (Fishbein et al. 2023), Taxonomy; tens of millions of microorganisms are known, spanning multiple domains (Sunagawa et al. 2015; Biteen et al. 2016; Proctor et al. 2019), such as Archaea (Thompson et al. 2017), Viruses (Carroll et al. 2018), Bacteria (Fournier and Gogarten 2010), and Protists (Gilbert et al. 2012), Functional genomics; there are billions of functional genes encoded by bacteria, Epigenetics; Compositions are influenced by various ecological and evolutionary factors (Gilbert et al. 2012; Liu H et al. 2019). All these areas of MDM can potentially improve our understanding about microbial population, evolution, and ecological function.

Artificial intelligence (AI) and big data analysis have made it possible to mine the MDM more efficiently to achieve a deeper understanding of the microbial world and its applications (Biteen et al. 2016; Zha et al. 2022). Millions of samples, each containing a few hundred megabytes (Mbs) of amplicon data were used to generate big microbiome data. Furthermore, whole-genome sequencing (WGS) data can provide about 10 gigabytes (Gbs) of data from a single microbial sample. As a result, metagenomics research with over 1000 samples might generate more than 10 terabytes (Tbs) of DNA sequencing data (Mallick et al. 2017; Knight et al. 2018). Microbiome big data may provide information for various applications, including identifying new genes and species, sample source monitoring, and phenotypic prediction (particularly for disease diagnosis).

Herein, first, we will discuss the latest advancements made in culture-dependent and culture-independent methods. Next, we will discuss the major MDM under investigation and the available bioinformatics approaches for their genome mining. The following material will first introduce the use of big data and AI technology, and at the end, we will review the advantages of AI in tackling MDM challenges and potential applications.

Historical background of microbial cultivation

Historically, the bacteria that were unable to grow on media used routinely in the laboratory were considered dead. Until 1982, the phenomenon of viable but non-culturable (VBNC) was introduced (Xu H et al. 1982). Xu and his colleagues (Xu H et al. 1982) first differentiated between viable and uncultivable bacteria. However, later several studies demonstrated different physiological states in bacteria (Kell DB et al. 1998; Bergkessel et al. 2016; Hegedüs et al. 2018). The VBNC bacterial cells maintain their viability but are unable to grow on standard laboratory media (Oliver 2005). Like other living organisms, microbes interact with various biotic and abiotic factors in an environment, triggering stress responses and survival mechanisms when the conditions become unfavorable (Hegedüs et al. 2018). Several groups of bacteria survive extreme conditions by forming spores or converting into a non-sporulating dormant state (Shoemaker and Lennon 2018). As per our understanding, VBNC is an adoptive characteristic of bacteria that helps them survive for a long time under sub-optimal growth conditions such as nutrient depletion, extreme pH and temperature, osmotic stress, exposure to heavy metals, increased salinity and UV radiation (Cunningham et al. 2009; Said et al. 2010; Li et al. 2014; Shoemaker and Lennon 2018). Although VBNCs are unable to colonize media and clearly distinguish form the dead cells. But, their prominent characteristics are metabolically active, carrying high ATP levels, preserving cell integrity, and retaining genomic/plasmid DNA (Oliver 2005; Li et al. 2014). Typically, metabolic activities and transport are significantly reduced in VBNC bacteria and their morphology, cell wall, and cell membrane compositions are modified (Bodor et al. 2020). In fact, the environmental factors responsible for VBNC lead to enhanced cross-linking within the peptidoglycan cell wall, followed by modification in outer membrane proteins or fatty acids in the membrane (Oliver 2010). Certain bacteria form coccoid-shaped or exhibit dwarfing in the VBNC state indicating a reduced surface area to volume (SA: Vol) ratio that helps to minimize energy consumption (Oliver 2005; Li et al. 2014). Nevertheless, morphological alteration can’t be used as the only distinguishing trait for the VBNC state (Oliver 2010). Different bacterial species have been reported to form VBNC cells using diverse regulatory mechanisms, but currently, our knowledge regarding the underlying complete molecular processes is still inadequate. However, two basic stress regulators including oxytocin receptor (oxyR) and RNA polymerase of stationary (rpoS) phase are critical in developing the VBNC state (Li et al. 2014). The gene oxyR is responsible to regulate other genes during oxidative stress. The protein RpoS is a sigma factor and general regulator of a stress response (Li et al. 2014). The RpoS gene is constitutively expressed during the VBNC state, while its lack leads to a fatal phenotype (Boaretti et al. 2003).

VBNC cells detection

Although the VBNC bacteria are live but are not capable of growing either on standard bacteriological solid or liquid/semiliquid media; therefore, the typical plat assay can establish the inability to form a colony (Zhao et al. 2017). Kogure and his colleagues demonstrated a procedure in which acridine orange elongated the VBNC cell when treated with nalidixic acid, a cell division inhibitor (Kogure et al. 1979). This enables us to observe non-dividing normal VBNC cells under a compound microscope. Moreover, several fluorescent dyes can be used to investigate the morphology of VBNC bacterial cells (Fakruddin et al. 2013). Besides that, counterstaining of bacterial cells with 5-cyano 2-3-ditolyl tetrazolium chloride and 4, 6-diamino 2-phenyl indole can be used to detect viable and total bacteria simultaneously (Ramamurthy et al. 2014). The BacLight viability kit uses two staining dyes, fluorescent CYTO-9 and fluorescent propodium iodide, which can diffuse through the damaged membrane of dead cells only, and thus unstained living bacterial cells can be estimated for a viability assay (Ramamurthy et al. 2014). Owing to these selective properties, the BacLight kit can also be used in a fluorescent microplate reader, fluorometers, and flow cytometers.

The molecular method exclusively targeting nucleic acid, such as qPCR, cannot differentiate dead cells from viable cells (Ramamurthy et al. 2014; Zhao et al. 2017). While combining qPCR with staining dye propodium mono azide, which crosses only injured cell membranes, binds to the nucleic acid and prevents amplification; therefore, it is capable of detecting the nucleic acid exclusively in viable intact cells (Zhao et al. 2017). A recent study suggested a novel method for detecting cell viability by mRNA of stress-related genes using RT-PCR (Oliver 2010; Ramamurthy et al. 2014). Since bacterial mRNA is only present in metabolically active cells and has a short shelf life. Thus, it not only differentiates between live and dead cells but, when used after an unsuccessful cultivation assay, may also be employed to detect VBNC bacteria cells. The methods for detecting VBNC cells often rely on specific conditions or assays, making it challenging to develop a universal approach. These methods may not capture the full diversity of VBNC states across different microbial species. Therefore, standardized processes are required for detecting and characterizing VBNC cells to ensure comparability across studies. Efforts should be made to establish consensus criteria for defining and identifying VBNC states. Moreover, research is necessary to elucidate the mechanisms governing the induction, maintenance, and resuscitation of VBNC cells. Understanding the factors influencing transitions between VBNC and culturable states is critical for accurate detection.

Scout theory and resuscitation of bacteria

In 2009, Epstein proposed a theory about the microbial life cycle and stated that a dormant bacterial population intermittently but stochastically and independently adapts to the specific habitat cues and reverts from the state of dormancy into active growth (Epstein 2009). The newly reactivated growing cells that revert to an active state in response to random and unknown internal events are called scout cells (Buerger et al. 2012a). The scout is not a genetic variant and does not alter from the normal growing cells of the community. Dormancy is a common natural phenomenon that assists survival under unfavorable and sub-optimal conditions (Buerger et al. 2012a). However, some points of the Scout hypothesis are controversial (Kell D 2009), and the precise molecular mechanism that regulates the Scout has yet to be elucidated. However, some studies support the Scout theory (Buerger et al. 2012b, 2012a). The study showed that the stochastic growth of bacterial scouts in a dormant bacterial population not only explains the random pattern of latent infection but also the mystery of why some culturable bacteria are found in all VBNC communities (Bogosian and Bourneuf 2001). A recent study proposed that gene expression stochasticity is a critical factor in microbial epigenetics that affects the transition state, including dormancy and scout formation, independent of the environmental stimulus (Xu Y and Vetsigian 2017). A scout utilizes the available resources and transitions from dormancy to scout status. When the conditions are favorable again, they secrete various signaling metabolites such as siderophores, autoinducers, resuscitation-promoting factors, and catalases. Otherwise, the original scout cells die, and a new cell arises as a scout in the dormant community (Bury-Moné and Sclavi 2017). In future, both environmental biotechnology and healthcare biotechnology are expected to benefit from the artificial development of dormant cells into scouts using intracellular triggering substances.

In contrast to dead cells, VBNC cells perform measurable metabolic activities that enable them to start growing; thus, the entire process is reversible. The word “resuscitation” was first coined by Roszak and his coworkers to describe the transition of non-culturable bacterial species (Roszak et al. 1984). A major challenge of resuscitation is to differentiate the residual, previously undetected culturable cells in the bacterial population from the original VBNC cells (Zhao et al. 2017). Pinto and his colleagues hypothesized that bacteria have a transition stage called the “resuscitation window,” which refers to the duration VBNC cells will resuscitate in response to a particular stimulus (Pinto et al. 2015). However, the exact mechanism and duration of the resuscitation in various bacterial species are yet to be explored. In addition, the resuscitation window in different bacterial species varies significantly. For instance, in the case of Salmonella enteritidis, the duration is four days, while for Micrococcus luteus it is lost for six months, but for Citrobacter freundii it is 11 years (Dhiaf et al. 2008). A more recent study demonstrated that a subpopulation of VBNC bacterial cells showed an improved revival capacity than the rest of the cells in the population (Wagley et al. 2021). The team quantified the fitness of Vibrio parahaemolyticus to become VBNC and subsequently resuscitated. They reported that lactate dehydrogenase (lldD) was upregulated in all subpopulations of VBNC cells. Moreover, the deletion of lldD led to a rapid phase shift to the VBNC state, whereas the addition of lactate to VBNC cells promoted resuscitation and extended the resuscitation window (Wagley et al. 2021).

As mentioned earlier, the VBNC state is activated by various environmental stress factors; neutralizing or eliminating these stresses may help them to regain culturability. Thus, optimum growth parameters vis-à-vis temperature, energy source, nutrient concentration (carbon/nitrogen ratio), chemical substances such as antioxidants, reactive oxygen species (ROS) scavenger, etc., and co-culturing with other appropriate species might enhance recovery of the VBNC cells (Ramamurthy et al. 2014; Zhao et al. 2017). Nevertheless, in the majority of cases, the resuscitation process is very complicated and can not be merely induced by removing the stress factors (Bodor et al. 2020). Various stimuli might stimulate the resuscitation process, including autoinducers, siderophores, and other factors (Kell D 2009; D’Onofrio et al. 2010; Su et al. 2014).

One of the significant challenges with the scout theory is the limited empirical evidence supporting the existence of specialized scouting bacteria within populations. The identification and characterization of these scouts remain elusive, raising questions about the practicality of this theory. Thus, understanding the factors and mechanisms involved in bacterial resuscitation is critical, as a controlled resuscitation of dormant bacteria could be exploited for practical purposes, such as in bioremediation or industrial processes.

Recent trends in the isolation of unculturable bacteria

Unculturable bacteria cannot grow under standard laboratory conditions due to the transition into dormancy or slow growth states. These bacteria may be considered “k-strategists,” meaning that they have slow growth due to minimal resource accessibility yet maintain a stable existence in their natural habitat. In contrast to these species, “r-strategists” are fast-growing and quickly respond to a consistent nutrient supply. Therefore, using standard laboratory growth strategies might result in the overgrowth of fast-growing “r-strategists” while veiling the slow-growing “k-strategists,” which are adapted to growth in a habitat containing minimal resources (Janssen 2009).

Based on this concept, an attempt was made to isolate the unculturable bacteria by mimicking natural conditions through decreased inoculum size, nutrient supply, and increased cultivation time (Davis et al. 2005). In addition, media modification and other abiotic factors, including temperature, pH and oxygen availability, secondary metabolite dependency, and synergistic interactions (co-culturing), were also considered while developing new cultivation approaches. Buerger and his colleagues suggested that providing the same growth medium to culturable but rare environmental species might cause them to escape isolation for a long time (Davis et al. 2005). Therefore, a longer incubation time favors isolating novel species due to their stochastic awakening features (Buerger et al. 2012b).

During the last decade, in situ cultivation has been attracting research attention. The in situ resuscitation of VBNC bacteria in response to the extracellular organic matter or recombinant Rpf from Micrococcus luteus was conducted (Jin et al. 2017). Upon quantification, the culture supernatant of M. luteus called extracellular organic matrix, mainly contained polysaccharides and protein (Rpf) at 405.7 mg/L and 25.1 mg/L, respectively (Su et al. 2014). Recently, Chaudhary and his colleagues developed a diffusion bioreactor to cultivate the unculturable bacteria in their natural habitat. They were able to detect an additional 35 previously uncultured species as compared to the traditional culturing method. A new culturing medium named intensive soil extract medium (ISEM) was utilized to efficiently isolate unculturable bacteria and several novel taxa, which accounted for 49% and 55%, respectively, of the total isolates detected. The ISEM medium was designed based on new soil extract using 80% ethanol (Chaudhary et al. 2019).

Nichols and his team developed an isolation chip (iChip) comprising hundreds of miniature chambers (Figure 1). Each small chamber was inoculated with one environmental bacterial cell. They revealed that bacterial recovery can be enhanced by significantly using the iChip compared to the standard cultivation approach (Nichols et al. 2010). The iChip technique provides a promising environment for culturing previously unculturable bacteria. However, the application of iChip could not be universally adopted due to the difficulty of ensuring single-cell distribution in each well. To overcome this drawback, an integrated flow cytometry-based fluorescence-activated cell sorting-isolation chip (FACS-iChip) was designed (Liu H et al. 2021). This approach was initially employed for paddy soil samples, and a comparative study was conducted with the standard cultivation approach. The authors revealed a retrieval rate of 25% and a recovery rate of ∼40%. In this study, up to 500 strains were cultivated from 19 genera using eight FACS-iChip plates, and 13 genera were found novel. These results indicate that FACS-iChip offers a promising and readily available system to explore MDM (Joshi and Chitanand 2020). More recently, the iChip design was modified to make it more suitable for isolating extremophiles from hot springs. A total of 107 bacterial strains belonging to 17 genera were isolated using a modified iChip. These constitute 25 strains that were found to tolerate temperatures up to 85 °C (Zhao et al. 2023). A new magnetic nanoparticle-mediated isolation method was developed to recover metabolically active cells of previously unculturable phenol-degrading bacteria from wastewater sludge. The author’s probe isolated bacterial cells from the sludge and showed that nearly 79% were successfully labeled. These results suggested that magnetic meditated isolation could efficiently recover live/functional cells from a complex microbiota (Zhang D et al. 2015).

Figure 1. Timeline showing advances in culture-dependent (a) and culture-independent (b) approaches to study microbial dark matter (MDM).

The advent of metagenomics

Before the turn of twenty first century, the study of microbes required to grow them in isolation. For over a hundred years, microbiologists have observed that the number of microbial cells detected under a microscope did not correspond well with the colonies on the plate. In 1985, Staley and Konopka hypothesized that only 1% of the microbial world could be accessed via standard plating techniques, and the phenomenon was termed “a great plate count anomaly” (Staley and Konopka 1985). This observation incited a question: how can we investigate the uninvestigated 99%? In order to respond this question, Handelsman and his colleagues coined the term metagenomics in 1998 and referred to “the collective genome of soil microbiota.” Handelsman and colleagues published a landmark study that demonstrated the feasibility of extracting DNA from a soil sample and analyzing the genetic material to identify the microbial species present in that sample (Handelsman et al. 1998). This was a significant breakthrough, as it allowed researchers to study entire microbial communities in their natural habitats rather than relying on cultured isolates. At the start of twenty-first century, the term metagenomics was introduced (refer to community genomics or environmental genomics), which involves the analysis of genetic material directly recovered from environmental samples (Handelsman 2004). Further, the broad application of metagenomics from ecology and environmental sciences to chemical industry and human health were highlighted (Handelsman 2004).

In 2004, the Global Ocean Sampling Expedition (GOS) was launched, which aimed to study the microbial diversity of the world’s oceans using the metagenomics approach. The GOS project generated a massive amount of sequencing data, providing valuable insights into the diversity and function of marine microbial communities (Venter et al. 2004). Since then, metagenomics has been applied to various fields, including Microbiology, Ecology, Environmental science, Biotechnology, and Medicine.

Over the following years, the development of high-throughput sequencing technologies, such as 454 pyrosequencing and Illumina sequencing, enabled researchers to rapidly sequence and analyze large amounts of DNA from complex microbial communities. This led to the emergence of the Metagenomics field, which has since grown rapidly and expanded to encompass a wide range of applications.

High throughput sequencing technology sheds light on MDM

Our planet is a diverse microbes reservoir and carries over 20 000 prokaryotic species (Louca et al. 2019). Among these, slightly over 2% have been validly described (Yarza et al. 2014), which indicates how little knowledge we have about this major branch of the tree of life. Before the 1970s, when DNA sequencing and computing technologies were not advanced, microbial research was based only on pure cultures. Since then, developments in sequencing and computational tools have enabled us to explore MDM from complex environments. Particularly, metagenome-assembled genomes (MAGs) and single amplified genomes (SAGs) obtained from metagenomics and single-cell genomic approaches, respectively, have been the most reliable strategy that enables the investigation of single-cell/organism level analysis of a complex ecosystem. The construction of MAGs is dependent on the shotgun metagenomic data, which employs computational binning tools such as MetaBat (Kang et al. 2019). On the other hand, SAGs are obtained by physically isolating single cells from a complex sample, followed by cell lysis, whole genome amplification, sequencing, and assembly (Kogawa et al. 2018) (Figure 2).

Figure 2. Schematic workflow used to generate SAGs and MAGs. Single-cell is physically separated from a complex environmental sample using microfluidics, flow cytometry, or a single-cell printer. Various genome binning tools are freely available, including MetaBAT (Kang et al. 2019), MaxBin (Wu et al. 2016), and CONCOCT (Alneberg et al. 2014). The quality of the draft genome is checked using CheckM or MDMcleaner and should follow the criteria for SAGs/MAGs suggested by the genome standard consortium (Bowers et al. 2017).

In 2004, Jill Banfield’s group attempted to construct MAGs. The authors successfully constructed several MAGs from biofilm samples taken from an acid mine drainage system. They constructed high-quality draft genomes and partial genomes of bacteria and archaea from environmental DNA using shotgun metagenome sequencing (Tyson et al. 2004). Genome analysis of the unculturable iron-oxidizing bacteria in the acid mine enabled the researchers to explore their genetic potential and survival strategy in extreme environments (Tyson et al. 2004). The Craig Venter group conducted the second large-scale metagenomics study of its kind using ocean samples. The group estimated that the samples comprised about 1800 species, of which 148 were identified as novel (Venter et al. 2004). These two studies opened exciting opportunities with the help of metagenomics. Nayfach and his colleagues recently utilized large-scale metagenomics data and recovered 52 515 MAGs of high and medium quality from diverse geographic locations. Further, they established earth microbiome catalogs using 10 450 high-quality MAGs (Nayfach et al. 2021). The earth microbiome catalog MAGs also contain 12 556 previously unidentified species, thus providing a comprehensive resource that captures extensive phylogenetics and functional diversity among uncultivated bacteria.

Similarly, a genome and gene catalog for glacier microbiomes was established using 2358 MAGs and 883 culturable strains from 21 Tibetan glaciers. In addition, over 25 million non-redundant protein-coding genes were prioritized for the glacier catalog (Liu Y et al. 2022). Stewart et al. assembled 4941 MAGs from 83 ruminant cattle and predicted a large set of rumen microbiome proteins (Stewart et al. 2019). Another large-scale study was established for the Arctic microbiome catalog, which comprised 526 species using 530 MAGs. Where 441 MAGs belong to previously unidentified species (Royo-Llonch et al. 2021). Other researchers reconstructed 60 664 MAGs from 3810 diverse human fecal samples and revealed 2058 newly identified out taxonomic units (OTUs). This represents a 50% increase in the known genetic diversity of human gut microbiota (Nayfach et al. 2019). The gut microbiota was also investigated in another study where 92 143 MAGs were generated from 11 850 gut samples, revealing 1952 novel unculturable bacteria species (Almeida et al. 2019). These candidate species carry hundreds of novel gene clusters and help to understand the taxonomic and functional landscape of gut microbiota better.

The metabolic potential of microbes can be reliably predicted with high-quality MAGs and SAGs as compared to function anticipated from phylogenetic affiliation on the basis of a marker gene (Bowers et al. 2017). Even low-quality draft genomes are enough to answer a few basic scientific questions, i.e. who and where they are and what they potentially do in their communities (Iqbal et al. 2021). This information has enabled microbiologists to design rational experiments to study the physiology and metabolism of the targeted organisms (Hatzenpichler et al. 2020).

MDM: a Crux of knowledge

MAGs’ and SAGs’ analysis has unraveled some great wonders of microbial diversity and aided key discoveries. In the last decade, numerous MAGs and SAGs have been obtained from complex environmental samples that are representatives of a major lineage (Parte et al. 2020). The candidate phyla registered before 2020 are discussed somewhere (Jiao et al. 2021), while the focus here is on the phyla registered within the last three years (Table 1). Among these, some innovative studies on MDM are also discussed below.

Table 1. Novel candidate prokaryotic phyla discovered through SAGs/MAGs.

Phylum name	MAG/SAG	Sample source	Country	Reference
Altiarchaeota	MAG	Estuary sediments	USA	(Seitz et al. 2016)
Bathyarchaeota	MAG	Qi’ao Island	China	(Meng et al. 2014)
Binatota	MAG	Unknown	Public databases	(Chuvochina et al. 2019)
Blackallbacteria	MAG	Deep terrestrial subsurface	USA	(Probst et al. 2018)
Brockarchaeota	MAG	Hot spring	China	(De Anda et al. 2021)
Caldatribacteriota	MAG	Hydrothermal sediments	USA	(Hahn et al. 2020)
Desantisbacteria	MAG	Crystal Geyser	USA	(Probst et al. 2017)
Eremiobacterota (WPS-2)	MAG	Not applicable	Public databases	(Ward et al. 2019)
Genascibacteria	MAG	Groundwater	USA	(He et al. 2021)
Geothermarchaeota	MAG	Hydrothermal Subsurface fluid	USA	(Jungbluth et al. 2017)
Gribaldobacteria	MAG	CO2-driven geyser	USA	(Probst et al. 2018)
Hadarchaeota	MAG	Not applicable	Public database	(Chuvochina et al. 2019)
Hadesarchaeota	MAG	Shallow salt Sediments	USA	(McGonigle et al. 2019)
Halobacterota	MAG	Not applicable	Public database	(L Bräuer et al. 2020)
Hydrothermota	MAG	Not applicable	Public database	(Chuvochina et al. 2019)
Sifarchaeota	MAG	Costal Rican sediments	USA	(Farag et al. 2021)
Dependentiae (TM6 revised)	MAG	Chiba	Japan	(Takemura 2022)
Lokiarchaeia	MAG	Microoxic lake	Romania	(Bulzu et al. 2019)
Micrarchaeota	MAG	Unknown	Public database	(Baker and Dick 2013)
Modulibacteria	MAG	Methanogenic sludge	Australia	(Sekiguchi et al. 2015)
Moisslbacteria	MAG	CO2-driven geyser	USA	(Probst et al. 2018)
Montesolbacteria	MAG	Ground water	USA	(He et al. 2021)
Ratteibacteria	MAG	CO2-driven geyser	USA	(Probst et al. 2018)
Thermoplasmatota	MAG	Sea water	Australia	(Rinke et al. 2019)
Tianyabacteria	MAG	Deep sea sediment	Federated states of Micronesia	(Cui et al. 2021)
Torokbacteria	MAG	CO2-driven geyser	USA	(Probst et al. 2018)
Wirthbacteria	Both	Unknown	Genome Institute’s IMG database	(Hug et al. 2016)
Zixibacteria	MAG	Aquifer sediment	USA	(Castelle et al. 2013)
Abawacabacteria	MAG	Suboxic/anoxic aquifer	USA	(Anantharaman et al. 2016)
Aminanaerobia	MAG	Termite gut	Costa Rica	(Warnecke et al. 2007)
Asgardarchaeota	MAG	Unknown	ATGT database	(Rinke et al. 2021)
Aureabacteria	MAG	Gold mine deep terrestrial subsurface	USA	(Momper et al. 2017)
Auribacterota	MAG	Antarctic lake		(Williams et al. 2022)
Baldrarchaeota	MAG	Rongcheng Swan Lake	China	(Liu Y et al. 2021)
Borrarchaeota	MAG	Rongcheng Swan Lake	China	(Liu Y et al. 2021)
Canglongiota	MAG	Yap trench	China	(Zhang X et al. 2022)
Cloacimonadota	MAG	Antarctic lake	Antarctica	(Williams et al. 2022)
Deferrimicrobiota	MAG	Boreal peatland	USA	(Begmatov et al. 2022)
Electryoneota	MAG	Antarctic lake	Antarctica	(Williams et al. 2022)
Freyrarchaeota	MAG	Rongcheng Swan Lake	China	(Xie et al. 2022)
Gerdarchaeota	MAG	Coastal mud and sediments	China	(Cai et al. 2020)
Heilongiota	MAG	Tap trench	China	(Zhang X et al. 2022)
Hermodarchaeota	MAG		China	(Liu Y et al. 2021)
Hinthialibacterota	MAG	Antarctic lake	Antarctica	(Williams et al. 2022)
Hodarchaeota	MAG	Antarctic lake	Antarctica	(Williams et al. 2022)
Kapaibacteria	MAG	Bioreactor ecosystem	USA	(Kantor et al. 2015)
Kariarchaeota	MAG	Rongcheng Swan Lake	China	(Liu Y et al. 2021)
Lernaellota	MAG			(Williams et al. 2022)
Lithacetigenota	MAG	Hakuba Happo hot spring	Japan	(Nobu et al. 2023)
Njordarchaeota	MAG	Rongcheng Swan Lake	China	(Xie et al. 2022)
Omnitrophota	MAG	Antarctic lake	Antarctica	(Williams et al. 2021)
Qinglongiota	MAG	Yap trench	China	(Zhang X et al. 2022)
Sigynarchaeota	MAG	Rongcheng Swan Lake	China	(Xie et al. 2022)
Thorarchaeia	MAG	Microoxic nich	Romania	(Bulzu et al. 2019)
Wukongarchaeota	MAG	Rongcheng Swan Lake	China	(Liu Y et al. 2021)

Williams and his colleagues explored the ecologies of MDM phyla in an Antarctic lake (Williams et al. 2021). Using metagenomic sequencing and analysis approaches, Williams and his colleagues investigated the microbial communities inhabiting an extreme environment and revealed the presence of previously uncharacterized MDM phyla in that lake, expanding our understanding of microbial diversity. They also revealed genomic potential of these novel phyla contribute to their ecological roles. In addition, this study revealed six MAGs belonging to Candidatus Auribacterota bacterium and functional annotation showed that they are sulfide and hydrogen-producing heterotrophs. One MAG belonged to Candidatus Hinthialibacterota, which was thought to be a facultative anaerobe with the potential to reduce nitrate to ammonia and had a specialized metabolic pathway for complex organic matter mineralization (Williams et al. 2021). Three MAGs belonged to Candidatus Electryoneota, which are facultative anaerobes capable of sulfate reduction (Williams et al. 2022). Hug and colleagues generated >1000 MDM genomes and sub-classified the domain Bacteria, i.e. it demonstrated a hyper-diverse group of MDM termed as Candidate Phyla Radiation, abbreviated as CPR (Hug et al. 2016). Park and his colleagues used 8000 genomes from >1500 metagenomic samples and created a more prolific version of the tree of life. In this study, they increased the prokaryotic diversity by up to 30% when they included nearly 20 new phyla of Archaea and Bacteria (Parks et al. 2017). Current research also focused on archaeal SAGs and MAGs to explore the origin of eukaryotic cellular complexity (Zaremba-Niedzwiedzka et al. 2017). Notably, the E3 (Entangle, Engulf, Endogenize) model for higher organism generation has been suggested based on the isolation of Archean Candidatus prometheo Archaeum syntrophicum, which is a provisional name for well-characterized but uncultured organisms (Imachi et al. 2020). These studies of MDM change our perception of the evolution and origin of life. Besides that, the metabolic potential of MDM deserves more scientific attention, which has revealed several wounders in the recent past. For instance, genes responsible for anaerobic methane oxidation and sulfate reduction were found in MAGs, indicating that these two features, typically carried by two different partner organisms of a syntrophy, are merged within a single organism (McKay et al. 2019).

A recent study presented groundbreaking findings on the diversity and evolutionary relationships of Asgard archaea, a group of microorganisms believed to be closely related to eukaryotes(Liu Y et al. 2021). This study utilized MAGs and discovered several novel Asgard archaeal lineages. Through phylogenetic analysis, the investigators elucidated evolutionary relationships between Asgard archaea and eukaryotes, thus providing important insights into the origin of key eukaryotic features. The study cited evidence for the presence of genes that could be associated with eukaryotic cellular processes in Asgard archaea, further supporting their potential ancestral relationship. The expanded understanding of Asgard archaea’s diversity and relationships with eukaryotes contributes to our knowledge of early cellular evolution and the complex interplay between different branches of life on Earth (Liu Y et al. 2021).

Another study focused on understanding the diversity and ecological roles of episymbiotic CPR (Candidate Phyla Radiation) bacteria and DPANN (Diapherotrites, Parvarchaeota, Aenigmarchaeota, Nanoarchaeota, and Nanohaloarchaeota) archaea in groundwater ecosystems. In this study, MAGs were generated to uncover the genetic makeup and functional potential of these elusive microorganisms from various sites. The study revealed a high level of site-specific diversity within the episymbiotic CPR bacteria and DPANN archaea, indicating their adaptation to specific environmental conditions (He et al. 2021).

Furthermore, genomic analysis of SAGs and MAGs opened our eyes to the microbial potential that can be used for advancement in bioremediation, biofuel production, biomedicine, and biotechnology. The function and versatility of MDM help us to better understand the world in which we live.

Current challenges in MDM research

With the increasing ease and availability of diverse DNA sequencing methods, vast genomic data can be produced for exploring MDM. Thus, the scenario is shifting from a data-poor past to a data-rich present. The genomic data in public genomic databases is continuously increasing, providing an opportunity to unravel more MDM. Therefore, the current research in microbiology should focus on how to employ big genomic data. In this review, five major challenges related to MDM are highlighted.

Cultivation of MDM

The 1^st major challenge is the cultivation of MDM. Cultivation provides an opportunity to uncover the physiological features of microbes and serves as a vital source of information for microbiologists. Despite tremendous advances in the field of metagenomics, isolation of pure culture of most microbes remains a continuing challenge. However, few efforts in recent years have led to improved isolation methods. Nichols and colleagues, as mentioned before, designed an isolation chip (iChip) to domesticate the uncultivable microbes. Subsequently, a teixobactin antibiotic was discovered using iChip technology (Ling et al. 2015). The co-culturing technique was also introduced parallel to iChip. Co-culturing is a cell culturing system where two or more than two microbial species are cultivated with some extent of interaction between them (Goers et al. 2014). A successful co-culturing system demonstrated promising outcomes and led to the upregulation of several previously silent genes in a monoculture system. For example, indolocarbazole alkaloid was produced by Streptomyces sp. MA37 using the co-culture technique (Maglangit et al. 2020). While in the monoculture system, the biosynthetic gene cluster was silent and did not produce the metabolite. In addition, various functional-based labeling and multi-omics approaches, including nanometer-scale secondary ions mass spectrometry (nanoSIMS) and micro autoradiographauto rediograph (MAR), offer a basic knowledge essentially required to test the hypothesis to link function to a specific lineage. During cultivation, the microbial interactions are often disregarded, which impedes the investigation the relationship of MDM with symbiotic and mutualistic allies. Hence, data-oriented networks such as functional networks by using multi-omics data and co-occurrence networks from marker gene sets will enable to design better cultivation strategies.

Ecological and biogeographical role of MDM

The 2nd challenge of MDM is to investigate their ecological and biogeographical role. Understanding the ecological roles, distribution patterns, and biogeography of MDM is a challenge due to their limited representation in environmental datasets. Developing sampling strategies that capture MDM diversity and implementing large-scale ecological studies to unravel their ecological significance are ongoing challenges. Liu and his team developed a catalog for glacier microbiota and also explored the functional diversity of bioactive compound synthesis and safety prospects. The study attempts to predict microbial diversity and function in the glacier ecosystem, providing a resourceful tool for bioactive compounds and the health impact of glacier ecosystems (Liu Y et al. 2022).

The exact function of genes

The 3^rd challenge is to extrapolate the exact function from the genomic data set. Functional prediction from the genomic information is, at best, only a probability rather than a fact. Furthermore, it is found that more than 50% of the genes from SAGs and MAGs are hypothetical and could not be assigned to any specific function which restricts our understanding about the function of MDM. Thus, in the post-genomic era, advancements are required in the research field related to gene function. In order to fill the gap between MDM and their gene function, increasing efforts are needed in cultivation approaches and computational biology. Metatranscriptomic and metabolomics strategies have been employed to identify the chemical pathways and gene expression to investigate the dynamics of the recalcitrant microbes (Jiao et al. 2021). These omics approaches can be coupled with other labeling approaches, such as MAR and fluorescence in situ hybridization (FISH), RAMAN, and nanoSIMS techniques, to unravel the assimilatory function of targeted microorganisms (Vandereyken et al. 2023). In addition, heterologous expression and synthesis of a certain functional gene could also be an effective strategy to evaluate function of the desired genes.

Candidatus taxonomy

The 4^th key challenge of MDM is taxonomic classification. MDM organisms often represent novel lineages that do not fit into existing taxonomic frameworks. Currently, due to the imprecise taxonomy and nomenclature, communication regarding MDM is difficult for scientists. The presently used international code of nomenclature of prokaryotes (ICNP) does not apply to un-culturable microbes (Hugenholtz et al. 2021). Additionally, SAGs and MAGs are leading to genome-based discoveries of phyla associated with Bacteria and Archaea. Therefore, this creates confusion and makes it difficult for the scientific community to communicate effectively. That is why a reliable and stable nomenclature system is instantly required to name the newly described SAGs and MAGs. Previously, the genome sequence of SAGs and MAGs was suggested to serve as type material data for taxonomic description and an independent naming system for MDM was proposed (Murray et al. 2020; Hugenholtz et al. 2021); however, none of them were adopted and recognized globally. Therefore, developing accurate and standardized methods for the taxonomic classification of MDM organisms is crucial. This involves refining and expanding existing taxonomic schemes and developing new approaches, such as utilizing genomic and phylogenetic information, to accurately classify and name these novel taxa. Hedlund and his colleagues developed a “Seqcode,” a code of taxonomy where genome sequence serves as a nomenclature type. According to this scheme, the code enable valid publication of prokaryotic name based on strain MAG/SAG (Hedlund et al. 2022).

Handling and analysis of big data

The 5th challenge is handling and analysis of MDM big data. Recently, microbiome samples from different regions of the globe have been rapidly increasing, which has resulted in a massive increase in DNA sequencing data. Several specialized databases are currently available for microbiome data integration, including MetaGenomic Rapid Annotations using Subsystems Technology, abbreviated as “MG-RAST” (Meyer et al. 2019), “MGnify” (Mitchell et al. 2020), and “Qiita” (Gonzalez et al. 2018), as well as general databases such as the National Center for Biotechnology Information (NCBI), Sequence Read Archive (SRA) (Bernstein et al. 2020). The public resource “MG-RAST” has been developed for automated metagenomic phylogenetic and functional analysis. It generates automatic functional assignments by comparing metagenomic sequences to protein and nucleotide databases. Similarly, “Qiita” is a web-based open-source tool that allows non-bioinformaticians to run their studies and meta-analyses readily. Among these resources, the “MGnify” database (Mitchell et al. 2020) is a common data resource for microbiome investigations, encompassing millions of microbiota samples and sequencing data and analytical findings connected with samples. Millions of microbiome samples, DNA sequencing data, and metadata have been submitted to these databases and majority of them are freely available, which is a primary source of knowledge discovery (Mitchell et al. 2020).

Currently, numerous microbiome data analysis and mining tools are available that may be utilized for various analytical methodologies and phases. For example, “Mothur” (Ye et al. 2019) is a well-known classical tool for quality monitoring of 16S rRNA sequencing data, “QIIME 2” (Bolyen et al. 2019) is another extensively used conventional method for characterizing the structure of microbial communities. Similarly, many additional analytical methods for the functional characterization of microbial populations are now accessible. These methods allow for the quick conversion of metagenomics data to community structure and functional characteristics (Knight et al. 2018). Furthermore, complex classical tools, such as “HUMAnN2” (Franzosa et al. 2018) and “MetaPhlAn4”(Blanco-Míguez et al. 2023), are available to enable in-depth investigation of microbial communities. Furthermore, the classic Bayesian-based method “Source Tracker” (Knights et al. 2011) may be used for microbial source tracing, while “antiSMASH” (Blin et al. 2021) and the AI technique “DeepARG” (Arango-Argoty et al. 2018) are employed to determine gene functions. Due to the unavailability of culture representatives of MDM, contamination in SAGs and MAGs not only obfuscates microbiologists’ views on the uncultured microbial majority but also leads to misleading comparative genome analysis. More recently, an open-source, python-based “MDMcleaner” has been launched to detect the previously unnoticed contaminations in MAGs/SAGs (Vollmers et al. 2022).

Addressing these challenges requires interdisciplinary collaborations, advancements in technology, and the development of innovative methodologies. Overcoming these challenges will pave the way for a deeper understanding of MDM organisms, their ecological importance, and their potential biotechnological applications. Alongside challenges, the study of MDM offers several exciting opportunities for scientific exploration and discovery.

Role of artificial intelligence (AI) in MDM data analysis

Currently, there are millions of metagenomic sample data from hundreds of worldwide locations has resulted in the insufficiency of AI approaches for mining huge sequencing data, either for sample comparison and source tracing or for functional annotation (Knight et al. 2018). The available approaches for genome mining, source tracking, and discovery of dynamic patterns are simple, either based on distance calculations or database searches which are unable to mine cryptic intrinsic patterns among thousands of samples. These deficiencies necessitate the development of an AI-based method that will not only aid to shed light on MDM but also help in knowledge discovery.

AI plays an increasingly important role in metagenomics, as it can help automate and streamline many of the complex data analysis tasks involved in studying microbial communities (Zou et al. 2019). Here, we briefly discussed the AI technologies used to explore the diversity and application of MDM.

Machine learning (ML)

ML plays a significant role in advancing microbiome research by offering powerful tools for data analysis, pattern recognition, and predictive modeling. ML algorithms can be trained on microbiome data to classify samples, and make gene function predictions. ML models can also be applied to accomplish tasks such as taxonomic predictiony of microbial species, identification of biomarkers, disease states or treatment response prediction, and functional profiling of microbial communities. The machine learning approach can also be used to mine the metagenomic data for the discovery of bioactive compounds (Huang et al. 2023). DL is a subset of ML that utilizes artificial neural networks (ANN) with multiple layers. DL models can extract hierarchical representations from complex microbiome data, enabling more nuanced and detailed analyses. DL has been used for tasks such as metagenomic sequence classification, prediction of antibiotic resistance genes, image-based analysis of microbial communities and novel antimicrobial metabolites from next-generation microbiome sequencing data (Ma et al. 2022). Among the ML methods, random forest, support vector machine (SVM), and linear regression models are widely used to analyze microbiome data (Pasolli et al. 2016). Recently, Wirbel et al. developed a machine learning-based tool, SIAMCAT, to analyze human microbiomes and identify disease-specific gene markers to predict health conditions (Wirbel et al. 2021). Ma and her colleagues use deep learning to analyze human microbiome sequence data and identify several antimicrobial peptides. To validate the deep learning models, the predicted peptides were tested in the laboratory and results showed that 11 peptides are inhibiting the growth of antibiotics resistant bacterial strains (Ma et al. 2022).

Convolutional neural networks (CNN)

CNNs are a type of DL model commonly used for image analysis. In microbiome research, CNNs can be applied to visualize and analyze microbial community structures, such as analyzing fluorescence in situ hybridization (FISH) images or fluorescence microscopy images of microbial cells. In addition, to determine host phenotype from microbiome sequencing data, a neural network-based tool was designed using a novel augmentation method (Lo and Marculescu 2019). They tested the suggested method on the datasets of hierarchical matching pursuit (Turnbaugh et al. 2009) and diseases such as adenocarcinoma and esophagitis (Marcos-Zambrano et al. 2021). More recently, researchers employed convolutional neural networks (CNNs) to analyze and classify gene sequences from metagenomic samples based on their potential to encode bioactive molecules. The model was trained on a large-scale metagenomic dataset and was capable of identifying gene sequences associated with antibiotic and anticancer activities (Melo et al. 2021).

Recurrent neural networks (RNN)

RNNs are a type of DL model that can analyze sequential data. In microbiome research, RNNs can be used to model temporal dynamics and interactions within microbial communities, such as analyzing time-series microbiome data or predicting microbial community development over time. CNN and RNN work like a neural network of the human brain. In AI, Neuran is a functional network that gathers and categorizes the sequencing data in a provided-specific manner. The network acts like a regression analysis of the statistical method. Artificial neural networks were designed, and microbiome sequencing data was classified into non-virus and virus classes. The operating characteristic curve was generated and showed an accuracy of 0.79, which was well beyond the chance level (Marcos-Zambrano et al. 2021).

Reinforcement learning (RL)

RL is a branch of AI that focuses on training agents to learn optimal actions based on environmental feedback. In microbiome research, RL can be used to optimize microbial community design, such as designing synthetic microbial consortia for specific functions or optimizing interventions for manipulating microbiota composition. Recently, scientists employed RL to optimize the design of synthetic microbial communities in the gut. They used an RL framework to guide the selection and abundance optimization of microbial strains to achieve desired functional properties in the synthetic community. The study demonstrated that RL can be utilized to engineer synthetic microbial communities with desired functionalities in the gut (Clark et al. 2021).

Bayesian networks (BN)

BNs are probabilistic graphical models that can represent probabilistic relationships between variables. BNs have been used in microbiome research to model complex interactions and dependencies within microbial communities and to infer causal relationships between microbiota composition and host phenotypes. In recent years, researchers developed a framework called MEDUSA, which utilizes BNs to infer gene functions and predict the metabolic capabilities of microbial communities. MEDUSA integrated various data sources, including metagenomic sequences, functional annotations, and known biological networks, to construct a global gut microbial gene catalog and infer functional profiles. The BNs in MEDUSA were used to model the relationships between genes, functions, and microbial taxa to predict the presence and abundance of specific gene functions in metagenomic samples (Karlsson et al. 2014).

Predictive modeling

AI can be used to develop predictive models of microbial community dynamics, such as how different environmental factors affect community composition and function. These models can help researchers to identify potential targets for environmental monitoring and management. Langille and colleagues introduced a predictive modeling approach called PICRUSt (Phylogenetic Investigation of Communities by Reconstruction of Unobserved States) for inferring the functional capabilities of microbial communities based on 16S rRNA marker gene sequences (Langille et al. 2013).

AI can be used to construct metabolic models of microbial communities, which can help researchers to understand how different metabolic pathways are interconnected and how they contribute to ecosystem function. Korem et al. employed metabolic modeling to investigate the growth dynamics of gut microbiota. The study utilized metagenomic sequencing data from healthy individuals and patients with Crohn’s disease to construct personalized metabolic models of the gut microbiota (Korem et al. 2015). AI can also be used to identify potential drug targets within microbial communities based on the analysis of genomic data. This approach has the potential to greatly accelerate the discovery of new antimicrobial agents (Melo et al. 2021). AI can be applied to analyze the microbiome data of individual patients, allowing for personalized medicine approaches to be developed. This includes the identification of specific microbial species or functional pathways that may be associated with disease risk or treatment response. Zmora et al. focused on understanding individual responses to probiotics based on the gut microbiota using AI. This study highlights the potential of microbiome data in guiding personalized medicine approaches, specifically in the context of probiotics (Zmora et al. 2018).

These AI learning methods can leverage the large and complex datasets generated from microbiome studies to extract meaningful insights, identify novel patterns, and make predictions. However, it’s important to note that the effectiveness of these methods relies on high-quality and well-curated data, as well as proper validation and interpretation of results in the context of domain knowledge and biological mechanisms.

Future opportunities and application of MDM

Unveiling novel microbial diversity

MDM is a large repository of hitherto undiscovered microbial diversity. Microbiologists may uncover and define new microbial lineages by analyzing MDM, which helps us comprehend the Tree of Life and the evolutionary links between various groups of organisms.

Exploring novel functional genes and metabolic pathways

MDM organisms frequently have distinct genetic characteristics and metabolic capacities. Investigating the genomes of MDM species allows for the discovery of new functional genes, enzymes, and metabolic pathways with biotechnological and commercial implications. These discoveries have the potential to lead to the creation of novel enzymes, bioactive substances, and biotechnological processes (Acinas et al. 2021). The discovery of new genes from MDM, especially from unexplored extreme habitats, is expected to impact biological knowledge significantly (Figure 3). For example, discovering new proteases and lipase enzymes with enhanced degrading activity, stability, and specificity will be a major development in the industrial enzymes market, which was 6.2 billion US$ in 2020 and is expected to reach 13.815 billion by 2027 (https://www.grandviewresearch.com/industry-analysis/industrial-enzymes-market). In addition, the major discovery of the new CRISPR-Cas system from the MDM (Burstein et al. 2017), now employed in genome editing, highlights the potential of microbial genes to augment the evolution of the pharmaceutical industry and research tools.

Figure 3. Schematic representation of microbial dark matter (MDM) research progress and future opportunities.

Understanding ecological roles and interactions

MDM organisms are anticipated to have key ecological roles in various settings. Understanding their ecological activities and interactions with other microbes can help us understand ecosystem dynamics, nutrient cycling, and biogeochemical processes. It can also aid in comprehending how MDM organisms contribute to the stability and functionality of microbial communities. Understanding the interactions between unknown species, nearby microorganisms, and their different environments is crucial to comprehending the ecological contributions of MDM. Recently, researchers constructed a tool to examine the ecological importance of MDM. They built networks containing and excluding unidentified organisms for each habitat (Zamkovaya et al. 2021). A notable decrease in degree and betweenness was seen for all settings when the unknown taxa were removed from the networks. Surprisingly, unknown taxa consistently ranked as the top hubs across all locations, indicating that MDM is essential to the ecological health of these communities.

Drug discovery and bioprospecting

MDM is a potentially undiscovered source of natural chemicals and bioactive substances. Exploration of MDM genomes and metabolic capacities may result in the identification of new antibacterial drugs and other medicinal chemicals (Vuong et al. 2022). This creates new possibilities for drug development and bioprospecting initiatives. Recently, the antibiotic teixobactin produced by previously unculturable bacteria Eleftheria terrae was discovered and exhibited promising activities against Gram-positive human pathogens. Further, the scientists did not find any mutants of Staphylococcus aureus or Mycobacterium tuberculosis that were resistant to teixobactin (Ling et al. 2015). These features of the antibiotics point to a way for designing antibiotics that are less prone to acquire resistance. Piel and his colleagues found that the sponges carry about 1000 types of bacteria and, on average, produce >11 bioactive compounds per entity. In order to find out which bacterium produces which compound, they sorted individual bacterial cells and sequenced their genomes. Surprisingly, they discovered that one unculturable bacterial type was responsible for producing all the bioactive metabolites associated with the sponge. When the genome sequencing data was received from their collaborator, Piel remarked, “I almost fell out of my sofa” (Lok 2015). This was the first evidence that unculturable microbes are promising to produce bioactive metabolites. More recently, Piel and colleagues discovered the antifungal metabolites aurantoside produced by an uncultivable Chloroflexi lineage (Kogawa et al. 2022).

Advancements in bioinformatics and computational tools

The study of MDM requires the development of advanced bioinformatics tools and computational algorithms. There are opportunities for developing new methods for taxonomic classification, functional annotation, and data integration, as well as improving existing analytical pipelines (Walsh et al. 2022). These advancements will not only benefit MDM research but also have broader applications in other areas of microbial genomics and metagenomics.

Recent advances in metagenomics and single-cell genomics have allowed researchers to begin characterizing these previously inaccessible microbial communities, opening up new avenues for understanding the diversity and function of the microbial world. Herein, potential applications of MDM research in various fields are briefly discussed (Figure 4).

Figure 4. Application of microbial dark matter (MDM) in various fields.

Biotechnology and bioremediation

MDM research has the potential to identify novel microbial species and metabolic pathways that could be exploited for biotechnological applications, such as the production of biofuels, bioplastics, and other valuable compounds (Maglangit et al. 2020; Zha et al. 2022). MDM research could be used to identify novel microbial species and functional pathways that could be used to remediate polluted environments, such as oil spills or contaminated soils (Bodor et al. 2020).

Disease diagnosis and treatment

MDM research could help to identify previously unknown microbial species or functional pathways that are associated with disease risk or treatment response (Park et al. 2022). This information could be used to develop new diagnostic tools or targeted therapies for a range of diseases. For instance, Hacilar and his team utilized ML based techniques to distinguish between inflammatory bowel disease and healthy people. The study includes microbiome data from 382 individuals, with 148 persons diagnosed with inflammatory bowl disease and 234 being healthy (Hacılar et al. 2018). Dysbiosis in various body parts is associated with disease conditions such as colitis. In this regard, a study by Zeevi et al. used the ML approach and integrated different parameters of 800 individuals, including dietary habits and microbiome to predict the postprandial glycemic response (Zeevi et al. 2015). This study showcases the potential of ML-driven microbiome precision medicine, notably in oncology, as indicated elsewhere (Veiga et al. 2020).

Evolutionary biology

MDM research could help to shed light on the evolution of microbial life on Earth, including the origins of complex microbial communities and the factors that have shaped their diversity and function over time (Angulo et al. 2019).

Planetary science

MDM research could also have implications for studying microbial life beyond Earth. By studying the microbial communities that thrive in extreme environments on Earth, such as deep-sea hydrothermal vents or acidic hot springs, researchers can gain insights into the conditions that might support life on other planets (Milojevic and Weckwerth 2020). As Mars analogs environments are some of the extrem habitats on earth. Recently, researchers examined spatial distribution of potential impact-transported microbes on the Martian moon. Their results indicated that about 70–80% of the microbes are dispersed over the moon’s surface and drastically sterilized due to the galactic cosmic and solar radiation (Kurosawa et al. 2019). The search for extraterrestrial microbes is an ongoing and exciting area of scientific research. New discoveries, especially from extreme habitats, continuously advance our understanding of the conditions that might help life beyond the earth. However, to date, there is no confirmed evidence of microbes or life on other planets.

Future perspective of MDM

Microbes are ubiquitously found anywhere in the world, and most of them cannot grow in conventional laboratory conditions. Therefore, their ecological role and capabilities are poorly characterized. In the above section, opportunity and challenges, we described five major challenges of microbial DM and proposed their possible solutions. First, we must develop and establish new cultivation techniques for uncultivated microbes. Besides that still, there is a need to investigate large-scale data sets for microbial DM from various environments, especially the extreme environment, by integrating different omics approaches such as metatranscriptomics, metagenomics, metaproteomics, and metabolomics. This will increase our understanding of microbial taxonomy and functional diversity. Further, the link between microbial species and their in situ function needs to be improved by integrating omics approaches with MAR, FISH, Raman and nanoSIMS techniques. In order to understand better and enhance multidimensional knowledge, microbial DM research needs the integration of various disciplines. Moreover, as discussed earlier, the currently available computational tools are inefficient for analyzing big MDM data. Therefore, artificial intelligence-based robust tools are urgently required. To tackle these challenges, microbial DM research centers can be established and equipped with state-of-the-art facilities to provide technical assistance to train next-generation microbial DM researchers. This will open the magic box of microbial DM filled with wonders. Microbial DM research is still in the infancy stage and will continue to surprise us with exciting and unanticipated answers about evolution and ecological function.

Conclusion

The SAGs and MAGs-based discovery of Bacteria and Archaea phyla indicate that our planet harbors numerous microbes still to be explored. These studies shed light on the diversity of microorganisms and giant viruses and explore novel metabolic capabilities of MDM. This is the time to think outside of the box and make scientifically and financially tough decisions. For instance, sampling from unexplored sites where life is unlikely, adopting novel methods to allow for high-throughput detection, and preparing and training young researchers’ minds to detect unexpected and unknown entities and to change their perspective. Furthermore, understanding the MDM is itself a challenge for microbiologists, and data mining with AI could be a useful strategy to understand and predict its ecological function. MDM constitutes new species, functional genes, and spatiotemporal patterns; their insight is fascinating. Future discoveries in this field would undoubtedly result in the identification of new principles and could be applied in various fields such as biomedicine, healthcare setup, biosafety, and environmental monitoring. Finally, we emphasize the five challenges, including unstable taxonomy, cultivation, biogeographical role, functionality, and big data handling challenges, which should be addressed soon. Overall, MDM is an emerging and challenging field, but it also offers an opportunity for a microbiologist to explore large data sets with the aim of better understanding life and finding a better solution for current global apprehensions in environments and healthcare. We expect that an in-depth understanding and exploration of the MDM will bring innovation to the field of Biotechnology and Microbiology.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Science Foundation of Oujiang laboratory startup fund under grant no. [OJQD2022005], Wenzhou, Zhejiang, China.