Virtual forests: a review on emerging questions in the use and application of 3D data in forestry

ABSTRACT

Digital 3D technologies are emerging methods for recording and visualizing forests. Therefore, it is not surprising that these technologies have seen many applications and developments in recent years. In this study, we conducted a comprehensive review of existing 3D technologies within the context of forestry and how they interact with users and stakeholders. We present a summary of the requirements, visualization, and application of virtual forests. This includes an overview of state-of-the-art 3D reconstruction and visualization tools, which have seen a major increase in interest in the past few years, as evidenced by a preliminary analysis on research keywords. Based on the reviewed studies, we present the current trend and emerging questions, as well as challenges in the field of virtual forests. Further, we discuss the identified trends and challenges related to data acquisition, along with existing and potential future interactions between the 3D data and more specific demands from the forestry sector. We conclude that the use of digital 3D data in forestry is on the rise and that such novel methods show great potential and merit further attention.

KEYWORDS:

• 3D
• visualization
• forestry
• review
• virtual forest

Introduction

The great importance of forest ecosystems is undeniable: they provide timber and valuable ecosystem services, such as clean air, clean water, and biodiversity, and they help mitigate climate change effects (Brockerhoff et al. 2017). Forest ecosystems are subject to change, be it from natural disturbances, management activities including harvest operations, or climate change (Overpeck et al. 1990). They are, however, slowly changing systems; the effects of decisions regarding their management are not always immediately visible, and alterations due to climate change are not directly experienced by humans (Lindner et al. 2014). The people managing forests today, including various stakeholders, are likely not the same people who will benefit from them decades from now. This creates a discrepancy in communication and complicates management. The advent of digital technologies has opened the door to the virtual representation of objects, such as trees and forests, as well as their development over time. Novel methods can be used to visualize ecosystems and represent them in immersive ways and could have great potential to support and enhance communication between stakeholders and decision makers in forestry.

Bringing new technologies into a rather traditional field, such as forestry, comes with many challenges and opportunities. While the use of new technologies for purposes such as game development, architecture, or construction has become more and more common, their use in forest and landscape management is still very limited (Hejtmánek et al. 2022). Additionally, forestry has unique characteristics that sets it apart from the fields mentioned above, and standard methods applied in other fields might not work here because the dynamic nature of forested ecosystems is crucial for an adequate representation of management options. This presents an interesting dynamic to the subject of virtual forests and how they are implemented and accepted.

The concept of virtual forest has a broad definition; indeed, any digitized data related to forests may be considered a virtual representation of the forest. In this paper, the term virtual forest includes both geometric and semantic factors. Geometry-wise, the emphasis in this paper is on the use of 3D data to create the virtual forest environment. This 3D data may be either reality-based or design-based (Remondino and Rizzi 2010; Yang et al. 2018). In reality-based methods, 3D measurement techniques such as lidar and photogrammetry are considered, while procedural modeling is an example of design-based 3D data. This geometric representation of the forest may thereafter be augmented by semantic information, e.g. textures and species information. The virtual forest concept, as discussed here, includes studies relating to the various interactions between these two factors and how they are used to fulfill the different needs of the forestry sector. Virtual forests are therefore 3D digital representations of forests that can be used for a variety of applications, such as research, training, gaming, and forest planning.

In order to develop a virtual forest based on real forest data, multiple steps are needed. First, the necessary data must be acquired and processed. The data can come from various sources, such as lidar (Calders et al. 2016), photogrammetry (Kalacska et al. 2021), digital elevation models (Uusitalo and Orland 2001), aerial photography (Bates-Brkljac and Dupras 2001), or traditional manual forest inventories (Huang et al. 2020). This data needs to be processed to be used in virtual forest applications (Szabó et al. 2012). Raw data from multiple sources must be combined and converted into commonly used formats for digital applications, e.g. 3D point clouds or 3D meshes. Finally, the application needs to be developed considering user-friendliness and functionality (Bozgeyikli et al. 2016).

As regards to data visualization, one novel technology is Extended Reality (XR). It refers to a virtual environment in which users may interact in an immersive manner (Milgram and Kishino 1994). This may be done by augmenting the real world with virtual content (Augmented Reality or AR), immersing the user completely in the virtual world with a certain degree of haptic capability (Virtual Reality or VR), or a combination of the two (Mixed Reality or MR).

To form a basis for the development and use of these technologies in the field of forestry and landscape management, we therefore aimed to identify questions of emerging interest. The purpose of this study was to fill knowledge gaps which hinder a wider application of such technologies in forestry research, practice, and communication between stakeholders. Mapping the various topics in virtual forests will help focus efforts on research and development and identify new opportunities.

Materials and methods

To ascertain the recent trend in the field of virtual forests, we conducted a simple search of the Clarivate Web of Science for several keyword combinations in the topics of digital and virtual forests. Figure 1 shows the number of search results for different keywords and combinations thereof. No restriction was made as to the fields of the searches. We considered not only the interest in the topic, but also how it is applied at the cutting edge of forestry and the key challenges in creating a 3D representation of a forest.

Figure 1. Bar chart describing the number of certain keywords related to virtual forest found on Clarivate Web of Science.

While almost 13,000 articles matched our search for “digital forest,” the topic “virtual forest” yielded only about one-fifth as many results (3,004). Both keyword searches showed an upward trend in the number of related articles over the last years. However, “digital forest” showed a much steeper incline. The keyword search for “digital twin” yielded many articles in an unrestricted search, but the number was drastically reduced when restricted to forestry, only 13 results (Figure 1). This indicates that while the concept of virtual forest is potentially an important and emerging topic of interest, not much has yet been published related to forestry.

From this first overview of the literature, several sub-topics emerged from the concept of virtual forests. First, the creation of a virtual forest environment has gained a lot of attention in recent years, concurrent with developments in 3D reconstruction technologies (Calders et al. 2016; Abegg et al. 2017; Mokroš et al. 2018). Lidar, or laser scanning, has seen a significant process of democratization as more manufacturers have emerged, thus creating a competitive market for both terrestrial laser scanners (TLS) and aerial laser scanners (ALS). The latter term is also often used interchangeably with the term lidar itself. Passive remote sensing has likewise seen important growth, with developments in photogrammetry and multi-view stereopsis (MVS), as well as passive sensors, such as multispectral and hyperspectral imagery. These novel data acquisition techniques are instrumental in promoting the notion of a virtual forest.

A second sub-topic emerging from the existing literature focuses more on the exploitation of the data generated by the first point for the purpose of creating a virtual forest. Novel visualization methods, such as VR and MR, have been used to represent virtual forests in an immersive manner. Other work in the literature involved using 3D data to create information systems akin to GIS (Geographical Information Systems), albeit by exploiting the information provided by the third dimension as opposed to the 2.5D approach used in traditional GIS. This is more evident in the case of urban forests, where 3D trees interact with an existing 3D GIS of the urban environment in various analyses related to micro-climate management. The third sub-topic identified relates to how the created virtual forest is used. The use of immersive technologies opens the potential for various types of uses, ranging from technical applications, such as remote forest monitoring and modeling, to human-related analyses, such as the effect of a virtual forest on users.

Results and analysis

This section will describe the following topics based on the results of our review:

• Requirements for virtual forests, where we discuss state-of-the-art methods for 3D data acquisition and processing, as well as other requirements for the use of 3D data within the context of forestry.

• Visualization of 3D forests, where we look at the creation and design of virtual forest applications, including the different available visualization platforms.

• Applications of virtual forests, where we then describe the most common applications of such digital forestry systems.

Requirements for virtual forests

3D data reconstruction techniques

Various 3D sensors and techniques are available today, aided by significant developments in both hardware and software. In general, these techniques can be divided into passive and active sensors, depending on the way they acquire spatial data (Granshaw 2020). For both passive and active sensors, a further distinction can be made based on the distance between the object and sensor: terrestrial, aerial (including drones and planes), and extraterrestrial (satellite-based).

One of the most common passive remote sensing methods is photogrammetry. In a wider sense, photogrammetry is a technique that can reveal 3D information from 2D images using optics and projective geometry (Schenk 2005). For 3D data, photogrammetry refers to the conversion of 2D images into 3D data. Images for photogrammetry can be acquired by different means, including drones (Frey et al. 2018), planes and satellites (Rupnik et al. 2018). Terrestrial close-range photogrammetry is also used frequently in various applications, due to its potential to deliver very precise and photo-realistic results, but it is less often used in forestry. This is due to photogrammetry’s limiting factors, which include the need for multiple overlapping images from various points of view and for a convergent network of images (Schenk 2005). The heterogeneous nature of forest terrain means that terrestrial photogrammetry is not a suitable method acquisition-wise, although it has been used for single-tree 3D reconstruction in some studies (Fol et al. 2022). In the near future, emerging low-cost sensors, such as fish-eye (Kükenbrink et al. 2022) and panoramic 360° (Hristova et al. 2022) cameras, may be contenders for TLS in forest applications.

Major developments in the computer vision domain in the past two decades have catapulted photogrammetry from a mainly aerial mapping technique to an important digital 3D reconstruction method. Advancements in image matching and camera pose estimation using the Structure from Motion (SfM) method (Wu et al. 2011) have enabled fast and automatic processing. Meanwhile, dense matching techniques such as Semi-Global Matching (SGM; Hirschmüller 2011) have made it possible for photogrammetry to generate dense 3D point clouds not unlike those created by lidar. More recent developments have increased photogrammetry’s 3D reconstruction capabilities further, such as the introduction of constraints derived from Artificial Intelligence (AI) in dense matching (Stathopoulou and Remondino 2020) and novel 3D structure generation paradigms, e.g. neural radiance fields (Mildenhall et al. 2021).

In terms of active sensing methods, lidar is the technology most encountered in the literature today. Lidar is a method for determining ranges (variable distances) by targeting an object or a surface using a laser beam and measuring the time required for the reflected light to return to the receiver. This laser sensing technology may further be divided into ground-based (TLS; Rehush et al. 2018) and aerial applications (ALS; Nex et al. 2015). In current parlance, the latter is often associated with the term “lidar” itself, while the former may also be known by the term “laser scanning.” Furthermore, technological developments in miniaturization have led to the invention of other types of TLS, such as mobile laser scanners (MLS; Kaartinen et al. 2012; Cabo et al. 2018) and even handheld solid-state lidar (SSL; Luetzenburg et al. 2021; Murtiyoso et al. 2021). Lidar is also characterized by the capability to acquire multiple returns. In aerial lidar, the first return is the highest object detected, while the last return, depending on the scenario, characterizes the ground surface. Full-waveform lidar records the complete waveform of each backscattered pulse (Mallet and Bretar 2009). It is thus possible to use this technique to generate both digital terrain models (DTMs) and digital surface models (DSMs), even in the presence of dense vegetation. This is particularly useful in forest scenes, as it generates a 3D representation of both the forest bed and the canopy (Dalponte et al. 2009). It is therefore used more often than passive sensors in forestry applications. Table 1 shows a categorization of these 3D techniques.

Table 1. Categories of state-of-the-art 3D techniques and example studies.

Method	Terrestrial	Aerial	Extraterrestrial
Passive remote sensing	Close-range photogrammetry (Fol et al. 2022), Fish-eye photogrammetry (Kükenbrink et al. 2022) Spherical photogrammetry (Hristova et al. 2022)	UAV/drone photogrammetry (Frey et al. 2018)	Satellite photogrammetry (Rupnik et al. 2018)
Active remote sensing	TLS (Rehush et al. 2018) MLS (Cabo et al. 2018) SSL (Luetzenburg et al. 2021)	ALS/lidar (Kükenbrink et al. 2021)	N/A

Notions on levels of detail and suitability of techniques

While various techniques exist for the creation of 3D data, the inevitable question is which method is preferable for forestry. The answer to this question, as is the case with most multi-modal geospatial applications, depends on the scale of the object in question. Earlier studies in forestry sub-divided this problem into area-based approaches (ABA; Næsset 2002) and individual tree detection (ITD; Hyyppa 1999; Kaartinen et al. 2012).

In the ITD-based approach, the 3D data is first segmented; this means that objects with similar properties are grouped together (Wang and Shan 2009b). Manual segmentation is possible but highly time consuming (Burt et al. 2019). Various methods have been presented to automate the segmentation process and derive metrics used for forest planning, such as stem location, tree height, stem diameter at breast height (DBH), stem density, and timber volume (Li et al. 2012; Vega et al. 2014; Hackenberg et al. 2015; Parkan and Tuia 2015; Burt et al. 2019). Recently, AI in the form of machine learning techniques has also been applied to help with this task (Mazzacca et al. 2022).

In light of the various sensors available on the market, a more detailed categorization of their use may be beneficial for future users. For example, close-range photogrammetry has been used to study specific features of trees (Mokroš et al. 2018), while the use of TLS, MLS and fish-eye cameras has been discussed as tools to support forest inventories (Kükenbrink et al. 2021). Meanwhile, Gollob et al. (2021) experimented with SSL sensors in the Apple iPad to derive tree parameters. Applications of unmanned aerial vehicles (UAV) and ALS generally involve ABA-based approaches (e.g. Sankey et al. 2017). Figure 2 is a summary of the relationships between the 3D techniques identified and their uses in the literature.

Figure 2. Categorization of 3D forest mapping techniques based on scene size and complexity.

In Figure 2, the X-axis divides the 3D data requirements based on the needs of the forestry domain. This division is partly inspired by a preexisting classification of level of detail (LoD) in urban mapping, standardized using the CityGML paradigm (Biljecki et al. 2016). The LoD concept, as implemented in CityGML, is based on geometric LoDs, which we attempt to emulate in Figure 2 – starting at the micro-scale and ending with very large scenes. Meanwhile, the Y-axis presents the complexity of the scenes in terms of geometric details.

3D data representation

In terms of 3D data generated by reconstruction techniques, several data formats may be encountered (Figure 3). The first format is the 3D point cloud, which is often, but not always (Nocerino et al. 2020), the direct result of the 3D reconstruction process, whether via passive or active sensors. The point cloud is therefore the most direct measurement of the scene and constitutes a high-quality voluminous representation of real-world objects (Rusu and Cousins 2011). A 3D mesh, on the other hand, is a form of surface reconstruction most often generated by triangulating a point cloud. The mesh can also be textured, either from interpolated colors from its original point cloud or from the projection of image pixels when the projective relationship between the 2D image and 3D model has been established (such as in most cases of photogrammetric application). Meshes have advantages in data handling, as they usually require a smaller data volume compared with raw 3D point clouds (Zerman et al. 2020). However, in some cases using a mesh involves a loss of the original detail, due to simplifications or interpolations, although this loss is usually very small (Park and Lee 2019).

Figure 3. Different representations of 3D data for trees: (a) point cloud, (b) 3D mesh using the Poisson method, (c) parametric quantitative structure model (QSM), and (d) stylistic design-based 3D model drawn using Blender (https://www.blender.org/accessed 12 June 2023).

While meshes still retain a level of fidelity to the original form of the object, parametric models sacrifice this in place of simplified primitives which are easier to integrate into mathematical models (Ochmann et al. 2016; Yang et al. 2018). Such primitives are often simple forms (e.g. cylinders, planes) that are efficient for parametric modeling and computation, whereas mathematical models may struggle to use point clouds and meshes as input. In tree modeling, the use of parametric quantitative structure modeling (QSM) is widespread (Hackenberg et al. 2015; Lau et al. 2018). Another option is design-based 3D modeling. These stylized 3D models emphasize aesthetic values to deliver realistic renders, and this form of modeling is therefore most commonly used in media such as films and video games.

Within the context of forestry, the choice of 3D data format is subjective and depends on the application. For example, point clouds and meshes are preferable for performing measurements and extracting tree species and other important forest parameters (e.g. DBH, clear wood), owing to their fidelity to real-world dimensions. On the other hand, parametric models may be more effective/appropriate for predictions about future forest conditions.

Regarding 3D data visualization for research and academic purposes, the development of XR applications is often done using a game engine, which provides tools and libraries to create and deploy the software. Popular game engines include Unreal (https://www.unrealengine.com/fr, accessed 10 March 2023) and Unity (https://unity.com/, accessed 10 March 2023). The file format required for 3D models in game engines is usually a 3D mesh or a point cloud, which presents advantages in terms of both rendering and visual aesthetics compared with parametric models. Interactive applications can be developed using these 3D model formats, physical processes can be implemented in the virtual environment, such as collision, gravity, light, and wind.

Several case studies regarding the conversion of point cloud data to meshes can be found in geology (Bonnaffe et al. 2007) and civil engineering (Wang and Chu 2009; Laksono and Aditya 2019). A method for processing point cloud data from natural environments was presented by Risse et al. (2018). They developed a workflow that can generate meshes from lidar scans with very high accuracy (millimeter range) for the visualization of ant habitats.

Typically, 3D models of trees created by lidar scanning, photogrammetry, or artistic tools are static. However, there are significant advantages to using dynamic models that are generated procedurally. Procedural modeling means that models are created using a set of constructive and generative rules (Ullrich et al. 2010). Dynamic tree models that are affected by environmental parameters can be used to show the influence of various environmental models and interactions on a forest scene. A vast set of 3D models is needed for the creation of compelling 3D scenes. Their development can be highly time consuming when done manually (Talton et al. 2011). In contrast, once procedural rules have been developed, the subsequent generation of models can be scaled up at minimal cost.

Procedural modeling could also be used to generate trees and forests. Because trees are biological organisms, their growth and appearance are determined by biological rules, forming a favorable foundation for attempts to recreate these rules for 3D models (Ullrich et al. 2010). Notably, as most of the methods currently available for the procedural generation of forest scenes are primarily used in gaming, their main objective is to create visually appealing results rather than ecologically correct ones.

A method to recreate a single 3D tree model with procedural methods was discussed by Stava et al. (2014). They used different types of static models as starting points, e.g. models from lidar scans, Xfrog (a procedural organic 3D modeler for plants; Lintermann and Deussen 1999), or SpeedTree (a vegetation modeling program; https://store.speedtree.com/, accessed 9 March 2023), as well as trees generated by Open L-Systems (a type of formal grammar to describe plant growth; Lintermann and Deussen 1999). Stava et al. (2014) created a model controlled by 24 parameters. The parameters were found by maximizing the similarities between the input trees and the generated trees. Controlling the models with parameters made it possible to create trees that were sensitive to their environment; for example, a tree standing next to another tree would look different than a free-standing tree. Populating virtual forests with procedurally generated trees not only makes it possible to create diverse and compelling 3D scenes but also provides a means to change the generated models using parameters.

Visualization of 3D forests

History of forest visualization

The visualization of forests has been a research topic for more than 30 years. Orland (1988) stated that video imaging (“cut-and-paste” or “paint” over digital photographic images) can be used to speculate about the visible consequences of different forest management practices. Since then, the goal of using computer simulations to visualize a forest stand, either as it currently exists or as it will look in the future depending on specific management activities, was mentioned in many publications related to forest visualization. However, what has changed over time is the visualization quality and fidelity and the degree of interactivity and immersiveness, which have developed alongside technical advances in modern computer hardware and software.

In 1990s, projects about forest visualizations used manipulated photographs (e.g. Orland 1994) or individual computer-generated 3D images from fixed key viewpoints (e.g. Buckley et al. 1998) and Bergen et al. 1998). These techniques were compared by McGaughey (1998), Uusitalo and Orland (2001), Tang and Bishop (2002), and Karjalainen and Tyrväinen (2002), while the effects of image quality were investigated by Daniel and Meitner (2001).

In the 21^st century, many studies have still involved the use of single images (Stoltman et al. 2004, 2007; Sheppard and Meitner 2005), sometimes with the possibility of changing viewpoints at “several frames per seconds” (Lim and Honjo 2003) or using animations based on predefined camera flight paths (Dunbar et al. 2004; Wang et al. 2006). A further approach involved a collection of omnidirectional photographs taken in a real forest, which then enabled the user to virtually (using a computer) walk through the forest on a predefined path (Abe et al. 2005). Interactivity with the forest visualizations improved in these years, leading to visualizations where flying through the forest was possible in real-time (Pretzsch et al. 2008; Cournède et al. 2009).

In subsequent years, the immersiveness of forest visualizations started to increase through the use of multiple screens to create a 135° (Stock and Bishop 2006) or 260° (Boukherroub et al. 2018) experience, or through the CAVE system (Fujisaki et al. 2008; Fabrika et al. 2018; Fabrika 2021). In the past five years, immersiveness was enhanced further by the introduction of head-mounted displays (HMDs; Astner 2018; Botev and Viegas 2020; Holopainen et al. 2020; Huang et al. 2021; Chandler et al. 2022).

In many of the most recent publications, a game engine was used for the visualization of the forest, a concept that was already proposed 20 years ago (e.g. Herwig and Paar 2002) but could not be implemented at that point, mainly for technical reasons. Besides the numerous recent publications about forest visualization, there is also at least one commercial virtual forest project, where VR can be used to evaluate forest treatment strategies in Finnish forests (https://www.metsagroup.com/, accessed 13 March 2023).

Traditional screen-based visualization

The visualization of 3D data is naturally conducted using digital screens, which are currently the most common conduit for digital data (Klippel et al. 2019). The earliest example of 3D data rendering using computers and a 2D screen display is the Sketchpad software (Sutherland 1964), which can be used to draw simple 3D primitives. In modern 3D rendering, the graphics processing unit (GPU) plays an important role. By using a dedicated GPU instead of a computer’s processor or central processing unit (CPU), the amount of rendered data can be increased significantly and rendering can be completed much faster (Palha et al. 2017). Two modern rendering engines are OpenGL (Johansson et al. 2015) and DirectX (Baek and Yoo 2020). Operations within the 3D space also require interaction with the GPU, with the two most common interfaces being NVIDIA’s CUDA and the open standard OpenCL (Ghorpade et al. 2012).

While greater 3D rendering capability has been the most prominent development in the last decade, due to demand from the entertainment industry, other 3D applications have also seen important developments, such as the rendering of point clouds. For example, the Point Cloud Library (PCL; Rusu and Cousins 2011) and open3D library (Zhou et al. 2018), both written for C++, enable users to benefit from GPU-aided 3D rendering of point clouds. In the Python language, open3D and the Point Cloud Processing Toolkit (PPTK; https://github.com/heremaps/pptk, accessed 10 March 2023) are two notable open-source examples.

Several ready-to-use software packages with user interfaces also exist. CloudCompare (http://www.danielgm.net/cc/, accessed 10 March 2023) is one example of a free software that has seen numerous applications in the 3D domain in general. Potree (Schütz et al. 2020) is another 3D visualizer which uses a web-based approach for portability. While these two examples have always been general-use 3D visualizers, other solutions exist which are more geared toward the forestry sector. For example, 3DForest (https://www.3dforest.eu/, accessed 10 March 2023) and SimpleForest (https://simpleforest.org/, accessed 10 March 2023) render tree point clouds while providing specific functions for their analysis, e.g. computation of the tree diameter or volume.

Modern immersive visualization

Figure 4 shows the major distinctions in modern immersive visualization technologies. In this regard, VR may be considered as a highly immersive way to experience digital forests. Users experience VR applications with a head-mounted display (headset) and can interact with the virtual environment via tracked controlling devices or hand tracking.

Figure 4. The differences between Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) within the context of virtual forests. AR adds a multimedia content into the real world, usually with minimal interaction with the environment. VR immerses the user completely in a virtual environment. MR mixes AR and VR by enabling users to interact with the real world, virtual world, and the multimedia augmentation simultaneously.

In contrast to conventional applications, VR provides the user with a deep absorption directly into the visualization (Milgram and Kishino 1994). In VR, users are brought into a simulated reality with which they can interact. This mimics interactions with the real world more closely than conventional 2D or 3D approaches. The sense of immersion comes from the experience of being completely inside the virtual environment, where real-world movement is translated into movement in the virtual world. The commercial focus in VR is mainly on gaming platforms; however, there is great potential for VR applications in a vast variety of fields in academia, education, training, and communication, including in forestry (Mantovani et al. 2003; Dede 2009; Kavanagh et al. 2017; Huang et al. 2020).

Unlike VR, AR and MR do not try to immerse users or separate them from the real world (Figure 4). Instead, in the case of AR, the real world is enhanced by adding and overlaying virtual features. In MR, the real and virtual worlds are blended (Liu et al. 2017; Çöltekin et al. 2020). AR and MR could thus potentially be used in the forest for overlaying data about trees in the forest, such as tree species, timber volume, and other metrics. Collectively, AR, VR, and MR are sometimes grouped as XR.

Even though forms of XR have existed for about 40 years (Burdea 2003), recent technical advancements and a general decrease in cost for consumer headsets have made the technology accessible to the public. In particular, VR has been reported to enhance engagement and learning (Dede 2009), and the technology has been applied successfully to the treatment of mental health issues, such as to combat-induced post-traumatic stress disorder and fear of flying (Lindner et al. 2017). Health improvements from forest visits have been explored extensively (Sonntag-Öström et al. 2014), and even virtual visits have been shown to improve wellbeing (Yu et al. 2018; Mostajeran et al. 2021). Finally, an immersive environment can provide a novel and intuitive way to look at complex 3D data (Matthews 2018), such as lidar point clouds from forest scans.

Developing an immersive 3D environment for XR brings challenges that differ from other types of digital applications. Entering a virtual world can lead to nausea and sickness when not executed properly, especially when there is motion involved (Hettinger and Riccio 1992). Ergonomics and user wellbeing should therefore be considered in the development of VR applications.

Modern game engines often come with out-of-the-box VR development capabilities (Jerald et al. 2014), but adaptations are necessary due to the wide range of hardware available. The creation of large-scale realistic 3D environments, such as forests, requires a lot of time and skill from the developers, as well as computational power. There is always a trade-off between good graphics and performance. This calls for a careful examination as to how much realism and detail are needed for a given application.

While these game engines basically employ the same rendering techniques as classical 2D screen rendering, the setup of the XR system enables an interactive environment for the user. The user may experience the rendered point cloud in real-time, although computationally speaking the 3D object remains static, thus not requiring a continuous render. This may change when the system is faced with larger point clouds, for example the lidar data from entire forests. The limited computing power of XR tools means that this is a particular challenge. Several studies have been conducted to try to solve this problem using various strategies, for example by harnessing the GPU’s full capacity or by subdividing the large data into quadtrees (Palha et al. 2017) or octrees (Kharroubi et al. 2019).

Applications of virtual forests

Virtual forests as a tool for education and training

Virtual forests could be of high value for education and training purposes. Visualizing the decisions made in the field makes it easier to assess the consequences of alternative decisions. This technique is ready for use, for example in marteloscopes, which have been installed all over Europe in the past years. A marteloscope is a finite forest stand with trees that are numbered, mapped, and surveyed. Training sessions are conducted by allowing people to walk through the physical forest and mark trees for cutting, identify possible habitat trees, or collect other relevant information (Krumm et al. 2019; Thormann et al. 2019). As a result, reports can be generated about the types of trees selected, tree diameter distribution, and other characteristics of the selections made, and individual choices can then be discussed with a group. For some marteloscopes, predictions of future development based on silvicultural models are available. However, so far the only way of presenting the resulting data has been via graphs and reports, and recently via 360° cameras. Connecting this approach to VR would allow people to experience the impacts of management interventions directly and in an immersive way, leading to a better understanding of forestry (Figure 5).

Figure 5. Illustration of a marteloscope represented in Virtual Reality (VR) using the Unity game engine (https://unity.com/accessed 12 June 2023).

Another application of VR is in forest engineering, i.e. in simulators. For example, VR could be helpful with manual activities, such as pruning for value enhancement using chainsaws, or in the logging profession. It is important that young people who start their profession as machine operators have the skills needed to operate the equipment safely and efficiently. For instance, harvesters and forwarders are highly automated, and operators need to be familiar with them to avoid accidents and to cut the trees in an adequate manner without damaging soils or remaining stands.

In Wang et al. (2009a), 3D parametric data were used to model optimal bucking operations. Meanwhile Wang and Shan (2009b) generated 3D representation of trees to represent forest stands and used the result as a decision-support system. In both of these applications, the 3D data used were parametric models generated from manually measured tree parameters. Similarly, Lin and Wang (2012) used this approach to optimize log processing, but mentioned that TLS data are beginning to be used and future integration of such 3D scanning method was planned. More recently, Pichler et al. (2017) employed drone and TLS-based survey to generate a virtual forest, before using it to help tree marking for timber harvesting.

Simulators are highly beneficial in training of forest machine operators (Ranta 2009). Harvesters and forwarders are highly automated, expensive hardware. Operators must be familiar with the machine to operate it safely and in an efficient manner. Simulators offer a stress free, controllable environment for gaining practice in the control of the machine and proper handling techniques. A study by Ranta (2009) showed that introducing simulator training clearly added value for harvester operator training. Moreover, continuous cover forest management is applied in many countries, which leads to diverse and structured forests that are also much more complex for machine operators; training simulators are therefore of high value in this setting.

Virtual forests as a tool for forest management

Visiting forests virtually opens several possibilities in forest management and operational forestry. In typical forestry practice, forest owners, foresters, buyers or sellers of timber, logging operators, and other stakeholders all need to make site visits to the same stands, often multiple times in a short time window. Here, accurate, virtual representations of forests could be shared virtually among the stakeholders, thus reducing the need for frequent in situ visits and increasing the level of information and awareness. Furthermore, the digital twin of the forest is always easily accessible. In Figure 6, a point cloud of a forest was visualized in a web-based user interface. The left map shows a canopy height model overlaid with detected individual trees, which are visualized as points, where color represents the tree species. The size of the point represents the tree dimensions in the mapped forest. For further information a tile-based 3D representation of the forest can be viewed as high-resolution 3D point cloud by clicking an area from the left map allowing users to see the digital twin.

Figure 6. 3D point cloud visualized in web-based User Interface (courtesy of PreFor Oy).

Virtual representations also make it possible to view forests from multiple perspectives and at different scales. This can help, for example, in planning logging operations where decision making occurs at several scales, from individual trees to the landscape. Different forest management and logging scenarios can also be compared virtually before any tree is cut, for example regarding how thinning intensity or thinning method affects the surrounding landscape.

Due to the geospatial nature of state-of-the-art 3D scanning technologies, another logical application for the data is in performing spatial analysis. Virtual forests may be used, for example, in planning timber hauling operations (Picchi et al. 2022) or determining the range of forest fire hazard (Jaiswal et al. 2002).

Virtual forests as a tool for communication among stakeholders

Virtual forests may also be used as a tool for more general communication. For example, Huang et al. (2021) used a digital forest environment to simulate climate change scenarios. Such applications could help diffuse key messages to larger audiences in a more immersive manner. Virtual forests have also been tested for therapeutic purposes, namely how immersing someone in a 3D-simulated forest environment may influence their stress level in the same manner as more conventional forest bathing or sylvotherapy (Mattila et al. 2020; Hejtmánek et al. 2022).

We also acknowledge the great potential of virtual forests to contribute to enhanced communication between stakeholders. Public debates about forest management and targeted ecosystem services have shown that there are multiple stakeholders, often having varying interests and targets but also different levels of forest expertise. For example, if oak is envisaged as a dominant tree species in the next generation, it is necessary to conduct intensive management interventions. These might easily be misinterpreted as clear cuts, which are often not accepted by the public. Visualizing the potential development of an oak forest under alternative management scenarios offers various stakeholders a way to visualize and better understand why a certain decision has been made. Considering the growing need for biodiversity and ecosystem services, it is becoming increasingly important to bring stakeholders together.

Discussion

Based on the literature study described in the previous sections, several interesting trends and challenges emerge. When looking at the reviewed work from a chronological perspective, a definite democratization of 3D recording techniques, and by extension the use of 3D data in forestry, can be observed. Indeed, in the span of around 10–15 years the accessibility of 3D technologies, such as lidar, has increased considerably. This has been aided considerably by the miniaturization of lidar sensors, which can now fit into more accessible and lower-cost drones. This miniaturization aspect has also played a role in TLS, albeit cost-wise no significant drop can be seen. Smaller and lighter TLSs, coupled with a more advanced registration process, has opened the way for the use of this technology in forests. Developments in MLS technology have also benefited the forestry domain by enabling a faster, almost real-time, data acquisition process.

The use of drones, and by extension photogrammetry, has also seen an important increase in the forestry domain. In this regard several studies have been focused on finding a low-cost solution to forest mapping (Mokroš et al. 2021; Fol et al. 2022; Murtiyoso et al. 2022). This push toward low-cost mapping has also been aided by the development of old and new technologies, such as spherical photogrammetry and solid-state lidar. The latter also highlights an interesting trend in forest data representation and communication: the preference for easy-to-use and user-friendly mediums, such as mobile phones. Indeed, many researchers prefer AR over other XR methods, i.e. VR or MR, for this very reason.

The use of 3D measurement techniques to create the forest’s digital environment has also shown a trend toward metric-based forest mapping, notably for national forest inventories. In turn, this geospatial data has led to efforts to create an integrated digital forest information system, in GIS or other forms. Data visualization continues to be done in a screen-based environment, as is usual for any 3D data, but a recent emerging trend can also be seen in the use of XR as an immersive alternative. At the moment VR seems to have the edge in terms of the volume of research dedicated to its use in forestry, but future migration to MR may be eventual if not imminent. Regarding point clouds vs meshes, at this point there seems to be agreement that each method has advantages and disadvantages and that the best option depends on the application. Researchers concerned with forest modeling may prefer meshes or geometric primitives, such as QSMs, for their lightweight nature, whereas point clouds remain popular for practitioners and users in general. In the future, the two data formats may be able to be combined and applied either individually or simultaneously according to the user’s needs. Table 2 presents a quick summary of a few examples of these applications as identified in the previous section.

Table 2. Summary of a few examples of the applications of a virtual forest identified in this study.

Applications of virtual forest
For education	For forest management	For communication
Virtual marteloscope Harvester simulator (Ranta 2009)	Bucking optimization (Wang et al. 2009a) Decision support system (Wang et al. 2009) Spatial analysis and planning (Pitchi et al. 2022)	Climate change simulator (Huang et al. 2021) Immersive platform for multi-stakeholder engagement

As is the case with data science in general, the use of AI is increasing in both interest and use. Indeed, the interaction between multiple disciplines will be inevitable in the future, including in the forestry domain. The concept of a virtual forest is already gaining traction, as demonstrated in this paper, which means that interaction with AI is only logical. AI is currently used mainly for data processing purposes, e.g. point cloud segmentation and object identification. However, judging from trends in other domains, there is a great potential for other applications, such as modeling.

The review presented in this paper also highlights several challenges for the concept of a virtual forest. While the concept itself refers to a system to support forestry applications, its implementation is very much reliant on other disciplines, with computer science and geomatics being the two most prominent fields identified in the literature. Despite the ongoing increase in multidisciplinarity, a definite gap still exists between the more distinct scientific disciplines involved. The identification of requirements in forestry and geomatics may involve different aspects, albeit equally important, which may generate confusion in an external observer. A concrete example is the definition of scale or level of detail. In this regard, a formal definition or even a standardization (as in Figure 2) may help to deliver the same ideas across different stakeholders. Admittedly this is not an easy discussion, and scientific debate on the important definitions are inevitable and necessary, but also welcome.

Regardless of the technical and scientific aspects involved in a virtual forest, from the perspective of a potential user and/or forest stakeholder the issue of acceptability is a challenge that was identified from the reviewed studies. A virtual forest may present a completely different paradigm to users who are more familiar with “hands-on” approaches. Visualization methods such as XR present a very interesting and vast potential for forest studies, but in the current state of technology they are not as widespread as previously forecasted, thus limiting user familiarity and, by extension, ease of use. The impact of virtual environments on user behavior, such as decision making (Oberdörfer et al. 2021), is another challenging research question that should be addressed to truly validate the advantages of migrating into a virtual environment.

Finally, the use of 3D technology in complex and heterogeneous environments presents a particular challenge. As described previously, a myriad of 3D technologies now exist to support the creation of digital forests, yet their applications are usually restricted to certain uses. Considering the rising popularity of forest 3D reconstruction, harmonizing the use of the different sensors involved and, more importantly, integrating and choosing the best sensor to deliver the best overall result are topics that we predict will be investigated extensively in the near future. The choice of sensors often depends primarily on area size, geometric quality and/or point cloud density, and sensor cost.

Conclusions

We identified an ongoing trend of increasing digital data use in forestry, specifically geospatially faithful 3D data. With this trend, an integration or at least standardization of the different paradigms, technologies, and stakeholders is inevitable. This is already the case in other similar domains, such as the AEC (architecture, engineering and construction) sector, as evidenced by the rising popularity of building information models (BIMs). The use of AI is another aspect that is spreading in the processing of 3D forest data, with no signs of slowing down. Regarding visualization, screen-based representation is still dominant, but immersive techniques such as XR are gaining traction. The main challenges identified are directly related to these trends: how to achieve an integrated system, and how such a virtual forest system will change interactions between forest stakeholders. While many advances have been made in recent years, numerous questions still need to be answered regarding the use of digital data and its use for the virtual representation of forests and forested landscapes. Addressing such open questions will be critical to ensure that these technologies can be used optimally by researchers, practitioners, and stakeholders in the future. However, this also means that the topic of virtual forests has a very large potential to serve the forestry domain by presenting data effectively and thus helping stakeholders make important decisions regarding our forests. This paper thus presents an attempt to systematize existing knowledge in the domain, and thus to help other researchers in understanding the current state-of-the-art of the use of 3D data in forestry.

Acknowledgements

We thank the European Union for funding the Horizon Europe project “Digital Analytics and Robotics for Sustainable Forestry” (Digiforest) [101070405].

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the HORIZON EUROPE Framework Programme [101070405].