The emerging potential of interactive virtual reality in drug discovery

ABSTRACT

Introduction

The potential of virtual reality (VR) to contribute to drug design and development has been recognized for many years. A recent advance is to use VR not only to visualize and interact with molecules, but also to interact with molecular dynamics simulations ‘on the fly’ (interactive molecular dynamics in VR, IMD-VR), which is useful for flexible docking and examining binding processes and conformational changes.

Areas Covered

The authors use the term ‘interactive VR’ to refer to software where interactivity is an inherent part of the user VR experience e.g. in making structural modifications or interacting with a physically rigorous molecular dynamics (MD) simulation, as opposed to using VR controllers to rotate and translate the molecule for enhanced visualization. Here, they describe these methods and their application to problems relevant to drug discovery, highlighting the possibilities that they offer in this arena.

Expert opinion

The ease of viewing and manipulating molecular structures and dynamics, using accessible VR hardware, and the ability to modify structures on the fly (e.g. adding or deleting atoms) – and for groups of researchers to work together in the same virtual environment – makes modern interactive VR a valuable tool to add to the armory of drug design and development methods.

KEYWORDS:

• Interactive virtual reality
• interactive molecular dynamics
• human computer interaction
• On-the-fly drug design
• head mounted displays

1. Introduction

Human perception, intuition, creativity, and expertise are central to computer-aided drug discovery and design (CADD) [1–5]. This is likely to remain so for the foreseeable future, despite recent significant and rapid advances in machine learning and artificial intelligence. The three-dimensional (3D) structure of molecules is essential to their function. An increasing aim of drug designs is to ‘escape from flatland’, i.e. to design and develop molecules with more complex 3D structure, which are found higher up the drug development pathway and tend to exhibit better solubility [6] and lower toxicity [7]. The human mind has evolved to visualize and understand 3D space, and to interact with and manipulate objects in this type of environment. In psychologist and computer scientist J. C. R. Licklider’s 1960 essay ‘Man-computer symbiosis’ [8], he postulates that humans and computers will develop a symbiotic relationship, where the different strengths will complement each other, e.g. human spatial reasoning and computer speed and accuracy. Virtual reality (VR) provides a means of bringing together human intuition with computational power and resources, and can be an effective interface between them.

Many CADD applications that involve molecular models allow modification of structures, and display structures or simulations of 3D biomolecules via two-dimensional (2D) interfaces such as a monitor, using controls such as a mouse, making it difficult to fully utilize human 3D intelligence. VR provides a platform for 3D visualization of complex biomolecular structures and their dynamics. With the addition of interactivity via using human hands and fine motor control to directly manipulate objects, VR provides ways to interact, such as modifying structures ‘on the fly’ (e.g. chemical changes) or directing simulations toward a solution visible to a human. Interactive VR is intuitive to use and enables humans to focus on the areas of the drug discovery process that benefit from human knowledge and perception, e.g. chemical intuition, visualizing chirality, predicting how proteins and ligands fit together, the effects of a conformational change, or which alterations to the chemical structure might improve affinity or specificity. This has significant potential tospeed drug design and development.

VR is the use of artificial sense stimuli and can extend to manual interaction with a computer program to ‘trick’ the human mind into both perceiving a virtual world and feeling embodied in that world [9]. It can allow for the addition of extra modalities, such as depth, touch, and sound, to enhance human interaction with computer software. To be fully immersed in VR is to feel disconnected from real-world stimuli, and to feel completely engaged with the simulated world [10]. Although VR has been around for many years [11–16], recent improvements to hardware and software have now made VR a technology that is ready for widespread scientific use. Advances include low cost, high resolution, fast refresh screens (for visualizing VR e.g. through lenses in a VR headset), fast graphics processing units (GPUs), high-level 3D graphics engines, and scientific VR software. Some examples of using VR in science and engineering for 3D visualization are: virtual restoration of archeological finds [17]; virtual exploration and cartography [18]; viewing the sea bed [19]; safety training in chemical manufacturing [20]; sport psychology [21]; telepresence for clinicians [22]; teaching anatomy [23,24]; investigating the molecular structures related to the SARS-CoV-2 virus [25] to name a few.

The promise of VR in drug discovery has been recognized for many years [26–28], and is now beginninig to be realised. VR offers several potential benefits over traditional molecular visualization tools, and over traditional interfaces for interacting with biomolecular simulations and molecular modeling. Firstly, VR allows the researcher to visualize drug molecules and their macromolecular targets in full 3D, which allows more detailed perception and deeper understanding of these complex systems, and so can informs design and modification of ligands in the process of structure-based design and development [29,30]. Secondly, VR allows interaction with molecules, primarily through VR controllers. The controllers can be thought of as a kind of ‘virtual pair of hands,’ allowing the user to grasp parts of a molecule or molecules as easily as if they were tangible, real-world objects. Thirdly, recent developments of interactive VR permit users to interact with a running molecular dynamics (MD) simulation the atomic level [31], allowing them to manipulate the system, modifying its structure and interactions ‘on the fly’ (Figure 1). Finally, some VR software allows multiple users to occupy the same virtual space [30,31] for collaboration and, e.g. for teaching (collaborative VR). In other words, modern VR technology (hardware and software) is not merely an update with better graphics: the addition of molecular motion via MD simulations, the ability to directly manipulate those simulations, to work together virtually, and to create and modify molecular structure from within the program turns VR from a visualization method to a research tool in its own right.

Figure 1. Interactivity for a protein-ligand system in the VR software Narupa IMD [31,32]. A) A protein-drug system (influenza neuraminidase complexed with the neuraminidase inhibitor oseltamivir) shown in the interactive molecular dynamics in virtual reality (IMD-VR) software Narupa IMD, where all atoms are rendered with the default ball and stick renderer. The VR controllers are also rendered in VR, where the orange circle represents the point of interaction with the controller (like a cursor for a mouse). B) Rendering of all atoms changed from ball and stick to lines and a different color scheme. C) The protein is here rendered in the ‘cartoon’ renderer and colored in rainbow, where red represents the N-terminal of the protein and violet represents the C-terminal. The drug is here rendered in the ‘cpm’ renderer and with atomistic colors to differentiate it from the protein. D) By grabbing single atoms of the drug molecule using each VR controller, the user can apply a force to coax the drug out of the active site. The force from the VR controllers is shown by the two light yellow sine waves between the controller and the drug molecule. E) The drug has now started to dissociate from the protein because of the interaction by the user in VR. F) A view of a VR user in real life, where the monitor is showing what the user sees in VR.

We use the phrase ‘interactive VR’ to refer to programs that allow the user to interact with and change structure of a molecular system, i.e. not programs where interactivity is limited to using the controllers to rotate, zoom, and translate the 3D structure to improve visualization. In this review, we discuss current VR hardware and some relevant VR software for structure-based drug design. We give examples of how interactive VR is currently being used to visualize and manipulate biomolecules, addressing features that are useful in terms of CADD. This is a rapidly developing field, and we highlight some pertinent recent applications, noting that this is not an exhaustive list of all VR-based activities in drug discovery; it provides a snapshot of capabilities of some current hardware and software.

2. VR hardware and software

There are many different types of VR hardware currently available, offering varying levels of immersion and interactivity. Low-cost solutions such as cardboard viewers that hold a smartphone in front of the user’s eyes offer a low-cost way of experiencing VR [32–34], however, the field of view is restricted in this case, and there is limited interactivity. At the expensive end of the spectrum lies the Cave Automatic Virtual Environment (CAVE), a room-size virtual environment first developed 30 years ago that tracks the user via CAVE controllers and glasses [35] (there are also state of the art VR systems that track the user’s entire body from head to feet such as Vicon Origin [36], but these are primarily used for entertainment applications so are not discussed here). A CAVE consists of multiple screens that project the simulated space onto either 4 or 6 walls. The user(s) experience(s) VR through stereoscopic LCD shutter glasses that display a 3D image and can also interact using CAVE compatible wireless controllers [37]. CAVE set-ups have been used for drug discovery in the past, however they are expensive, costing hundreds of thousands of dollars for a high-end set-up [38,39], and can have problems with immersion if the glasses do not cover the peripheral vision as well as problems with ventilation and claustrophobia [40]. Head-mounted displays (HMD), such as the Oculus Quest 2, HTC Vive Pro, and the Valve Index, are generally used with the VR software discussed in this review. The advantages of using HMDs are that: (i) they are now relatively cheap (between $300-$1,600 USD at the time of writing for all the necessary VR equipment, not including the computer) [41–43]; (ii) they are portable and lightweight; (iii) users experience a wide field of view (ranging between 90 and 130 degrees in the horizontal dimension); (iv) they have good spatiotemporal resolution; and (v) the experience can be fully immersive.

Typically, HMDs allow either 3- or 6- degrees of freedom (DoF). HMDs with 3-DoF allow tracking of rotational motion about the x, y, and z axes (known as pitch, yaw, and roll), but do not track translational movement. Examples of 3-DoF HMDs include the Oculus Go, Samsung Gear VR, and Google Daydream. This is the simplest kind of tracking and is achieved by sensors built into the headset. HMDs that allow 6-DoF track both rotational and translational motion in the x, y, and z directions. 6-DoF HMDs may require external trackers positioned within the room, while the newer HMDs, such as the Oculus Quest 2 and Oculus Rift S, are capable of 6-DoF through sensors and image processing within the headset. This kind of tracking approach seems to be the most common with the latest HMDs, suggesting this is where the industry is heading. The advantage of utilizing 6-DoF in VR is that the user can exploit their full range of motion to explore the simulated environment, by walking, crouching, and bending to achieve the best view.

There are several articles that discuss and compare the specifications of the most common HMDs in detail, including resolution, refresh rate, and pixel density [44,45]. For biomolecular modeling and manipulation, a commodity HMD is often the most affordable and practical solution; current examples at the time of writing include the HTC Vive Pro [46], Valve Index [47], and Oculus Quest 2 [48]. Facebook recently renamed itself to Meta [49], after the ‘metaverse’ (first coined in the novel Snow Crash [50] as an online VR world accessible and manipulable by users), highlighting an emphasis on their VR range, Oculus. Valve [47] owns the gaming platform Steam™ and is therefore likely to maintain their headset line so as to be able to continue with their core business, and HTC has been in the area for a long time having made collaborations with large corporations. All three of these headsets are reasonable choices, however we recommend that the user checks whether their HMD is compatible with the interactive VR program they wish to use. Different programs mentioned in this review offer different capabilities. Some allow topological changes to the protein or ligand, some offer different renderers, some allow interaction with a physically rigorous MD simulation directly, from within the VR environment.

3. Tools for interactive VR: controllers, gloves, and hands

There is a natural desire to reach out and touch simulated objects when they are presented in an immersive setting. VR controllers, such as those which can be bought as a set with HMDs, provide the user with a virtual ‘pair of hands.’ The user can interactively and intuitively grasp and manipulate structures as if they were physical objects in the real world. This allows a more extensive exploration of structures than manipulation on a 2D screen [51] as it is facilitated by the 6 degrees of rotational and translational freedom and the depth perception afforded by VR (although it should be noted that attempts to improve visualization of 3D molecules have been made without the use of VR, e.g. stereo viewers and 3D printing of molecules [52,53]). Compared to using a mouse to interact with the simulation, which only permits a single interaction at a time via a mouse click, some VR software allow interaction from two controllers per user simultaneously, resulting in greater control over the simulation. This can be important when exploring large molecules or studying an area of the system that may be non-trivial to navigate e.g. a buried binding cavity.

However, there is still the limitation that a VR controller does not have the same range of motion as a real pair of hands, provided by the flexibility of human thumbs, fingers, and wrists. The development of VR gloves allows the incorporation of the complex motions that hands are capable of. There have been some advances in VR gloves specifically for molecular manipulation (Figure 2) [54].

Figure 2. VR Gloves and their application [54]. (A) Pinch sensing gloves. The tracker is placed on top of the glove and the interaction occurs through a pinching motion between either the thumb and index finger, or thumb and middle finger. (B) Two users in VR interacting with a real-time MD simulation of a polyalanine peptide. A pinching motion allows the users to grab an atom and interactively maneuver the protein around the VR environment. Image reproduced from [54] licensed under CC BY-SA.

Hand tracking is an alternative approach. Molecular Rift [55] is an example of VR software that is focused on manipulation of biomolecules in 3D space using hand tracking rather than VR controllers. Their gesture recognition software [56] allows basic structure manipulation such as rotation and translation of the molecule through a series of predetermined hand gestures. While hand tracking has the potential to be the most instinctive way for humans to manipulate molecules in VR – it removes the need for using specific hardware – a focus group of Molecular Rift users found learning the gestures to be a lengthy process as some were not immediately obvious [55].

An exciting direction is to incorporate haptic feedback (i.e. the modality of touch) into the virtual world [31,57,58]. Specialist haptic systems have already been developed [59–65], however they can be very expensive or not easily available to the consumer. Haptic feedback in VR has been explored for use in surgical simulations [66–68], where a prospective surgeon can, for example, feel the resistance of body tissue beneath their simulated surgical tool. As this kind of physical sensation is well defined, it is relatively easy to mimic in a VR setting. Haptic feedback in terms of molecular simulation presents a different type of challenge [69–71] because there is no reference for what kind of feedback is ‘realistic’ or intuitively useful when manipulating these objects. On the other hand, there is the possibility of exploiting a phenomenon known as ‘pseudo-haptics’ to ‘feel’ molecules in VR. This phenomenon is where the user experiences the illusion of feeling things within the simulation [71]; there is no actual physical force. A recent experiment showed that users of the interactive VR software Narupa IMD (see below) could distinguish molecular properties using this pseudo-haptic feedback [72]. The study involved users interacting with an interactive molecular dynamics (IMD) simulation, where three Buckminster fullerene (C₆₀) molecules were simulated in the same virtual space, all with different bond force constants. The force constants affected how rigid the molecule appeared in VR: a large force constant meant the molecule was stiff and inelastic, and a small force constant meant the molecule was more flexible. Most participants were able to notice a difference between how the molecules behaved in the simulation, and correctly rank the C₆₀ molecules according to ‘elasticity’ (bond force constants). There is potential for the sensation of pseudo-haptic feedback to be exploited for drug design. For example, an IMD-VR user could estimate the barrier to dissociation of a drug from a protein based on how ‘easy’ or ‘difficult’ it feels to remove the drug from the binding site.

4. Applications of VR relevant to drug discovery

Although this review focuses on interactive VR, non-interactive VR of biomolecular structures clearly has uses in drug discovery, as noted above. VR provides depth perception which improves how the molecules are perceived in 3D over 2D screens and stereoscopic representations. As the screens are directly in front of your eyes, VR offers the advantage of being able to view molecules as simply as moving your head and with an increased field of view (compared to a screen or using stereoglasses), making the technology excellent as a graphical display.

The developers of ProteinVR [29] (an example of a protein viewed in ProteinVR is shown in (Figure 3.4) tested how visualizing a protein-ligand complex in ProteinVR compares to the 2D molecular viewer, VMD [73]. The complex chosen (T. brucei RNA editing ligase 1, known as REL1, and V2, a naphthalene-based inhibitor) has a binding pocket that lies in a deep, narrow cavity. The structure of the complex used had been generated using the automated docking program AutoDock Vina [74]. The users found that, by viewing the molecule in VR, it allowed intuitive exploration of the narrow binding pocket cavity by the user moving their head [29]. They hypothesized that the AutoDock Vina pose may be incorrect and suggested interactions which could be more favorable to binding [29]. This is an excellent demonstration of how users can utilize their spatial awareness in VR to note irregularities in the structure that might otherwise go unnoticed on a 2D screen.

Figure 3. Scenes from VR software for molecular modeling. 1) A scene from the interactive molecular dynamics tutorial from the Nanome software [30], where the user is inspecting the first frame of a trajectory. 2) UnityMol software [103] showing a cartoon and hyperball representation of the influenza neuraminidase protein (PDB 3TI6). 3) Hydrogen bonding (rendered as white glowing interactions) between a 20 amino acid long polypeptide in the software, Peppy [88]. 4) The SARS-CoV Mpro (PDB 2Q6G) in a rainbow cartoon representation using the web-based VR software ProteinVR [29]. Images created by the author using interactive VR software Nanome [30], UnityMol [103], Peppy [88], and ProteinVR [29].

Molecules are fundamentally dynamic [75], (and protein dynamics is often important for function) and therefore it can be important to consider molecular motion when designing drugs, e.g. to examine how a protein and drug candidate bind and interact with each other over time. MD simulations [76–79] are increasingly widely used in drug design and development. Popular programs currently used for visualizing molecular dynamics trajectories include VMD [73], Pymol [80], and Chimera [81]. However, these typically project 3D motion onto a 2D screen, so information is lost. Interactive VR software such as Nanome [30] and ChimeraX [82,83] can be used to visualize MD trajectories (Figure 3.1). Features such as the ability to play, pause, rewind, fast-forward, slow down, and speed up the simulation, which are available in traditional molecular viewers, are also available in these VR software packages.

As with standard screen-based GUIs, it is useful to visualize molecular properties and interactions such as hydrogen bonds and the electrostatic potential of molecules in VR. Interactions such as hydrogen bonds, salt bridges, and charge-dipole interactions contribute greatly to protein structure and stability, and ligand binding [84,85]. Drug binding relies on favorable interactions between the drug and the receptor, therefore viewing and understanding these interactions is crucial to drug design and development [86,87]. Many interactive VR programs such as Molecular Rift [55], Nanome [30], and Peppy [88] include a toggle for switching hydrogen bond visualization on and off. By viewing these interactions in 3D, the user can utilize their own depth perception to inspect the distances and angles between interacting groups. This is valuable for suggesting where modifications could be made to the drug or protein to generate additional hydrogen bonds. Molecular Rift and Peppy offer a striking visualization of this type of interaction, portraying the contact between the donor and acceptor atoms as a bright, dynamic cloud (Figure 3.3). In contrast to a flat dashed line, which is often used to indicate a hydrogen bond in 2D molecular visualization software, this makes it obvious to the user when a hydrogen bond is present because it is visually distinct from the rest of the biomolecule. Nanome (version 1.22.1) also provides the option to measure the distance and angle manually in VR, by clicking on the measuring tool and physically selecting atoms of interest. The value of the distance or angle is then displayed above the selected atoms. Finally, the ability to simply move your head and inspect the molecule from a different angle may be helpful when looking at hydrogen bonds (or other non-covalent interactions) between atoms buried below the surface of the molecule, where the structure is more labyrinthine. Recently, interactive VR (specifically using Nanome) was used to generate structural analogues of a small molecule with the aim of creating a SARS-CoV-2 main protease (Mpro) inhibitor [89]. The SARS-CoV-2 Mpro is a promising drug target for combating COVID-19, and it is indicative of recent progress that interactive VR is being used for drug design and modification purposes for this and other COVID targets.

In recent years, the pharmaceutical company Novartis has recognized the potential for interactive VR for early-stage drug research [90]. Using Oculus and Nanome, scientists at Novartis are utilizing interactive VR to investigate structural insights for potential drugs to combat COVID-19. US biotechnology company Nimbus Therapeutics have also deployed Nanome for drug discovery, stating that they expect the technology to ‘save tens of thousands of dollars per year’ and drastically accelerate the drug design process [91]. In December 2021, Roivant Discovery, the drug discovery division of biotechnology company Roivant Sciences, deployed the largest platform of Nanome to date [92]. By employing this interactive VR technology, dozens of scientists across the USA can collaboratively modify and process molecular data, allowing more informed decisions to be made and, ultimately, speed up the drug discovery process. Nanome has also been used for research into psychedelics by biotechnology start-up Psilera Inc., where scientists are investigating how natural psychoactive compounds may treat mood disorders and neurodegenerative diseases [93]. Nanome was founded in 2015, and is currently working with more than 15 biopharmaceutical companies in the US [92]. A recent partnership with Japanese information and communications technology company Fujitsu is allowing Nanome to provide their platform to Japanese pharmaceutical companies [94]. This impressive interest from the pharmaceutical industry indicates the potential that interactive VR has for early-stage drug development.

Nanome is also being used in education. Correspondence between the authors and Prof. Shozeb Haider at the University College London (UCL) School of Pharmacy helpfully revealed that he and colleagues at UCL are using Nanome for teaching and research, and have found interactive VR to be a ‘very easy and intuitive [tool] to teach students how structure-based drug design’ and that feedback from students has been ‘overwhelmingly positive’ (2022 e-mail from Prof. Haider to the authors; unreferenced). He also stated that ‘Multi-site discussions have been quite productive in our research group during the lockdown’ (2022 e-mail from Prof. Haider to the authors; unreferenced), highlighting the importance of the collaborative aspect of interactive VR.

In addition to visualizing electrostatic interactions, it may also be important to view the overall electrostatic potential of the molecule(s), as this can aid predictions in how and where a drug may bind to a receptor [95]. The visualization of electrostatic potentials using computer graphics was introduced by Weiner et al. in 1982 [96], who proposed that this kind of visualization would be beneficial for drug design. Indeed, the visualization of electrostatic surfaces has proven to be useful for investigating protein-ligand binding for CADD [97–99]. Software such as adaptive Poisson-Boltzmann solver (APBS) [100,101] and PDB2PQR [102] facilitate electrostatic calculations of biomolecules for visualization.

The developers behind UnityMol (example of interface shown in Figure 3.2) have created a VR platform for simultaneous calculation and visualization of electrostatic properties of molecules, named UnityMol-APBS [103]. The advantages that this programme has over APBS with a 2D viewer is that the user (i) has increased depth perception and field of view for examining the molecule, and (ii) can prepare APBS input files from within the virtual interface, using a graphical user interface (GUI) to send the input files directly to APBS tools. By simply reaching out, selecting the atoms of interest, and then clicking a button in the GUI, the user can seamlessly perform calculations within VR using the interface with APBS. The combination of the virtual interface with the APBS toolkit significantly reduces the time needed to manually prepare input files, which can often be verbose and rely on the user having prior knowledge of the software’s language and formatting style. Here, the need for learning a completely new software program is removed and replaced by a few easy gestures.

To investigate the utility of VR for exploring electrostatic surface potentials, the UnityMol-APBS developers examined a particular enzyme (Torpedo californica acetylcholinesterase, known to operate quickly due to its electrostatic properties) and a substrate molecule [103]. In VR, with a large field of view and ability to translate, rotate, and scale the size of the molecule, it was immediately clear from the cluster of animated electrostatic field lines – which represent the electrostatic potential gradient – where the active site was situated. The immersive nature of VR also provides a sense of spatial awareness, meaning the user can grab the substrate and translate it as a rigid body through the binding cavity. In this way, users can get a sense of how feasible the pathway is. It should be noted that, interactive VR is being used as a tool for drug research and development; that is not to say that the same results could not be obtained without the interactive VR element. Rather, pharmaceutical companies and research groups are using interactive VR to aid the drug development process, particularly in the early stages, and see value in this approach.

5. On-the-fly modification

Drug design has been described as both a science and an art [104], and therefore, creativity and ingenuity are integral to success. A study investigating the relationship between creativity and factors relating to creativity (such as flow of work and attention) of individuals tasked with a clothing design challenge found (via EEG measurements of brainwaves) showed that a VR environment allows participants to focus more on the challenge at hand. Users reported that the ability to walk about and observe their creation from different angles inspired new ideas for improvement, compared to designing their product with a pencil and paper [105,106]. Furthermore, the ability to interact with complex 3D structures in VR inspires creative discussion around what interactions contribute to drug binding, and what could be altered to improve these interactions [30]. Many programs used in CADD are focused on building and structural modification of small drug candidates and proteins. As the orientation of functional groups can be vital in terms of drugs binding, performing complex 3D modeling on a flat screen could lead to some error. Performing these modifications ‘on-the-fly’ in a 3D space could allow immediate answers to hypotheses and thus drive intellectual discussion.

The function of many proteins relies on the structure they adopt. The capacity to perform structural modifications of proteins and ligands could be enormously powerful [107]. The interactive VR chemistry educational tool Peppy allows such modifications: a cohort of undergraduate chemists used it to perform a range of 3D modeling tasks: from creating short chain polypeptides to constructing α-helices and β-sheets with complex hydrogen bonding networks to teach them the fundamentals of protein structure [88]. This study showed that students using interactive VR were enthusiastic and creative when building and scrutinizing polypeptides. Some students also reported that interacting with polypeptides in VR cleared up previous misconceptions they had from learning protein structure in 2D [88]. The developers of Peppy state that, while their primary aim is teaching secondary structure of proteins to undergraduate students, they also recognize its potential use in research [88]. They also highlight that the goal of designing a tool capable of on-the-fly modifications and mutations was to encourage engagement and creativity, as well as inspiring a deeper exploration of biomolecular structures than a 2D builder would allow. A limitation of this software is that it does not currently support larger proteins (currently, the maximum is 32 residues) [88]. Nanome allows modifications of much bigger proteins [30].

The ability to make on-the-fly modifications to both the internal structure (such as rotating functional groups) and the chemical structure (such as mutating a functional group) allows rapid testing of hypotheses and inspires creative thinking, as previous studies of GUIs etc have shown [105,106]. Nanome allows building and modifying drug-like molecules within a VR environment [30]. Researchers were able to build a small molecule inhibitor from scratch in the active site of an RIP2 kinase (PDB 5W5O), achieving an RMSD of 1.8 Å with respect to the crystal structure [30] (around 2 Å is considered by docking programmes to be a sensible cutoff for finding the ‘correct’ pose [108]). Furthermore, starting from the crystal structure, users were tasked with making chemical alterations to the drug, where the objective was to form additional hydrogen bonds with the enzyme to achieve a more tightly bound pose [30]. This shows that, not only were users of Nanome able to produce sensible drug-protein structures similar to crystal structures, but they could also make predictions for more favorable interactions and trial them directly [30]. Nanome also provides an energy minimization feature that can remove human induced artifacts, such as unrealistically long chemical bonds, and return a more energetically reasonable structure. In another test case, a VR session was held where four Nanome users (chemists) were tasked with pointing out sites for macrocyclization of an active starting compound [30]. The aim was to preserve activity of the compound while improving physicochemical properties. Within minutes the users had become accustomed to the software and were discussing how they could alter the structure they were given. The group were successful in creating structures with improved physicochemical properties, whilst retaining activity.

DNA nanostructures pose an interesting avenue for drug development as they can be used for targeted drug delivery. Vivern (Visulization, Interaction, and Virtual EnviRoNment) provides an interactive VR platform for users to build and analyze DNA nanostructures [109], including many on-the-fly modification tools, such as a ‘DNA un-twister’ and a ‘Magic Scale Lens’ that allows atomic level inspection of the DNA structures. In a recent user study, experts from the DNA nanotechnology field were tasked with using Vivern to build DN superstructures from scratch and analyzing existing structures [109]. The experts found that, from within the VR environment, it is much easier to spot mistakes and rectify them, which can of course be crucial from a drug design perspective.

The interactive VR tool Narupa Builder (Figure 4), an application of the interactive VR framework Narupa, also allows the creation and manipulation of molecules. Figure 4 shows a drug structure (oseltamivir carboxylate, an influenza neuraminidase inhibitor) and the steps taken by a user to create the drug in an interactive VR environment. While this small drug was generated atom by atom, Narupa Builder also has a built-in library that contains common molecular fragments, such as amino acids and aromatic rings, that allow the rapid building of larger molecules. There is also an energy minimization feature, like Nanome, and the ability to export the resulting structure to a MOL2 format. The Narupa Builder is freely available for download on Windows via https://irl.itch.io/narupa-builder.

Figure 4. Constructing oseltamivir using the Narupa Builder. 1) The structure of the target drug (oseltamivir carboxylate, commonly known as Tamiflu, an influenza drug) to be built using Narupa Builder. 2) Building the first carbon atoms of the 6 membered ring. The green axes represent the possible sites for a new atom to be bonded to the carbon. 3) The carbon backbone of the drug, without any hydrogens. 4) Changing from a carbon atom to an oxygen atom to be added to the drug. 5) Creating the carboxylate functional group. 6) Changing the bond order between two carbon atoms from a single to a double bond. 7) Finally, adding hydrogens to all the possible sites of hydrogenation. 8) The resulting structure.

6. Interactive molecular dynamics in virtual reality (IMD-VR)

One of the most exciting recent developments in this area is the combination of interactive molecular dynamics simulations with virtual reality (IMD-VR) [31], allowing researchers to literally reach into their MD simulations and orchestrate the pathway of molecular motion [110]. IMD [111–116] has some similarities to steered MD (SMD) [117], which uses small forces applied over a long simulation time to simulate atomic force microscopy (AFM) experiments or biological structural change. IMD differs as large forces are applied by the user, and the simulations are very fast to run at the cost of a decrease in accuracy [118]. In IMD, the user steers the system to drive rare/high barrier events such as protein conformational changes and ligand binding. Various scientific problems have been tackled with IMD including: ligand binding [119], protein engineering [120], polymers [112], colloids [113], transport through membrane proteins [121], fitting X-ray crystallography [116], education [122,123] and even small ~(40 atoms) ab initio quantum mechanical calculations [124]. However, performing complex 3D tasks with a 2D interface can be challenging. Although IMD was around during the early VR era [112], the breakthrough year was 2001 when Stone, Gullingsrud, and Schulten [111] interfaced the graphical display capabilities of VMD with the molecular mechanics/dynamics engine NAMD [125] and contemporary others did IMD in Java [113]. Interactivity was recognized as important, and since interaction with the simulation is via applied forces, further work in this area concentrated on adding haptic interfaces [111,118,120]: for example, force-feedback gloves combined with MD simulations via IMD were used in VR to design proteins for their stiffness and mechanical properties [120]. Some groups combined IMD, VR, and haptics [119,121], however, such frameworks only allowed one user at a time to interact with the simulation.

The IMD-VR software Narupa IMD [126] allows users to interact with MD simulations (utilizing the OpenMM physics engine [127]) on-the-fly in a fully immersive VR space. The purpose of OpenMM here is to propagate the MD; that is, the user provides force that moves the atoms, and OpenMM numerically integrates Newton’s equations of motion and provides the new positions of the atoms. For every timestep, this process is repeated to provide a trajectory that an IMD-VR user has effectively manipulated. Thhe user can interact directly with molecules in VR, manually applying forces that change the simulation. It should be noted that other force engines can be used with Narupa IMD. For example, electronic structure programs can be used for studies of chemical reactivity. In a recent human-computer interaction (HCI) study [51], participants were tasked with performing a range of molecular manipulation assignments (threading methane through a nanotube, changing helical screw sense, and tying a knot in a protein) using either a mouse and keyboard, a touchscreen, or IMD-VR. The participants were able to complete these complex 3D tasks faster using IMD-VR and with a higher success rate. IMD-VR has also been used as an undergraduate chemistry teaching tool, where students were tasked with performing small chemical rearrangements and docking tasks [128]. Feedback from students suggested that visualizing molecules in an interactive VR setting is more engaging than the traditional computer and mouse set-up they had previously used, and allows them to grasp chemistry concepts more quickly.

Automated docking of small molecules, fragments, and potential drugs is a popular CADD method due to its capacity to rapidly predict binding poses of small molecules to a target receptor molecule [129–132]. This provides insight into both the structural orientations (binding poses) of the small molecules as well as the corresponding binding affinities. Docking flexible molecules still poses a challenge for CADD, and human intervention is often required. Deeks et al. investigated the use of Narupa IMD for flexible, human driven, protein-ligand docking [133]. The experimental protocol involved testing whether expert and novice Narupa IMD users were able to interactively guide ligands (benzamidine, oseltamivir, and amprenavir) in and out of the binding pockets of three viral enzymes (trypsin, neuraminidase, and HIV-1 protease, respectively), and recreate their respective crystallographic protein-ligand binding poses (within 2.15 Å RMSD of the crystal structure) in 5–10 minutes of real time. A bound structure from the interactive MD was then extracted and a short amount of MD was performed to test whether the structures created with IMD-VR were stable. By utilizing human ability at 3D spatial manipulation, after only a short amount of time in VR (40 minutes, including time spent learning how to operate the VR equipment), non-expert users (most of whom were also not experts in drug docking) were able to produce docked structures of these three viral enzymes with FDA approved drugs [133].

In a separate study [134], expert Narupa IMD users generated structures of a small molecule inhibitor and an 11-amino acid oligopeptide substrate complexed to the SARS-CoV-2 main protease (Mpro), an enzyme involved in replication of the virus that causes COVID-19 [135]. Various protocols were tested during the IMD-VR docking process, including running the simulation with and without backbone restraints on the enzyme. The expert users were able to produce structures that replicate experimentally observed crystal structures [134]. The results emphasized the importance of formation of important hydrogen bonds and led to the recommendation to focus on forming such interactions to create stable bound structures. The expert users were also able to interactively dock an inhibitor and a substrate to an apo form of the Mpro, following the same procedure of forming key hydrogen bonds [134]. The docked structures generated from apo Mpro remained stable in MD. A further study that compared docked structures of peptide-Mpro complexes generated from AutoDock CrankPep (a popular automated docking program), Pymol, and IMD-VR showed that bound structures from all three methods were in good agreement with each other [136], which was encouraging.

Narupa IMD can save frames of the trajectory generated by the user. For example, if a user were to interactively grab a drug bound to an enzyme and pull it out into the solvent space, the user can save the frames of this trajectory using OpenMM, just like any other standard MD engine would allow. In a matter of minutes of real time, users can simulate a rare event such as drug unbinding [133,134,136]. Capturing these rare events may otherwise require a large amount of computational time and resources, as well as expert knowledge of the biological system prior to simulation. IMD-VR can allow users to sample a substantial range of the rugged energy landscape of a protein in an intuitive manner, using human spatial and chemical intuition to guide movements. IMD-VR allows manipulations on the atomic scale in order to observe how molecular systems evolve in real-time. This could provide an interesting way to explore the effects of a perturbation: e.g. when a drug binds to a protein, users can observe how the protein responds. It should be noted that the forces that a user exerts on the simulation may be high (this is a non-equilibrium simulation) and may cause significant perturbation. This should be considered when generating IMD-VR trajectories.

Currently, one important limitation of Narupa IMD is the number of atoms in the system that can be simulated and rendered at an appropriate frame rate (which of course depends on the available hardware). While it is possible to visualize several hundred thousand atoms for a static structure (where no MD engine is used, just observing the structure, e.g. an LSD molecule bound to a membrane-embedded HT2B human serotonin receptor using Narupa, see https://vimeo.com/420035802), dynamically rendering the positions of all these atoms as an MD simulation propagates is not currently feasible due to limitations of processing power. Clearly, this depends on the hardware being used for the simulation. aAdvances in hardware will improve issues with framerates. Similarly, the limitation of processing power and number of atoms also means that explicit solvent simulations are not yet generally practical using Narupa IMD. In all of the Narupa IMD applications referenced here [51,128,133,134,136], implicit solvent was used in order to ensure the framerate was reasonable.

IMD-VR has great potential in drug design and development. As discussed above, the ability to observe how a molecule responds to a perturbation could be useful in drug discovery, in understanding lgand binding affinity and kinetics. For example, drug bindinig to a protein may require a confromational change such as loop opening: such prrocesses can be driven intuitively by the user using IMD-VR to generate binding pathways. Being able to simply reach out and create binding poses and pathways of drug binding on-the-fly, using human spatial and chemical intuition, will help in drug development. Also transformative is the demonstrated potential for groups of researchers to work together virtually on molecular design and modeling problems.

7. Conclusions

Interactive VR software will not replace other CADD methods; many of the features described in this review are already available in non-VR biomolecular manipulation software and have been useful for a long time. Instead, interactive VR offers an alternative way of modeling and manipulating biomolecules and can be used in conjunction with other CADD methods. For example, it may be useful to use VR to visualize a docked structure created from automated docking, employing the depth perception and immersion to examine results and deepen understanding of the structure. Additionally, the interactive nature of the VR allows exploration of the structure in an intuitive manner, using simple gestures to grab and manipulate the system easily.

We encourage readers to try out these VR programs themselves. The ease of use of modern interactive VR programs, the immersion that they allow in 3D molecular space, and the ability to ‘touch’ molecules cannot be described adequately here; it is necessary to experience them. Despite the wealth of anecdotal evidence of VR tools improvement over comparable non-VR visualization and manipulation programs [93,137,138], there is a pressing need for well designed and rigorously tested user studies and HCI experiments to quantify this improvement. In this review we have only mentioned VR; as headsets shrink in size, weight and cost, the related technology of augmented reality, AR, (VR with an optical pass-through such that the user sees both the virtual and real worlds simultaneously) will grow in popularity. It is easy to see how both collaborative VR and AR could be invaluable tools for teaching and research collaboration, e.g. for distance learning. The only modalities discussed at length here were vision, manual interactivity, and touch; there are others that can be added to VR. For example, InteraChem [122] is an interactive VR program built on Narupa IMD-VR for use as a teaching tool. It includes an interesting modality of atomic ‘happiness’ that encodes the energetic feasibility of a particular bonding arrangement and it is represented by emojis drawn on atoms. This approach co-opts the part of the brain that deals with human social interaction and makes high-level concepts incredibly easy to understand. Temperature feedback (of controllers or gloves) has also been suggested as a possible method of information communication in IMD [120]. Color is part of how we see the world and can be used to encode information easily and intuitively. VR also includes the ability to play sounds and here extra information can be passed to the user. There is room for significant creativity in including these aspects here. How does the sound of a molecule’s vibrations change as it goes through a transition state? Getting good answers to these and similar questions, and developing the user experience, will allow interactive VR to be even more intuitive for human users. The easier the technology is to use, the more it can harness human creativity efficiently.

8. Expert opinion

Interactive VR is an emerging area that offers tremendous potential in drug design and development. State-of-the-art VR software and hardware can harness interactivity to manipulate and modify biomolecules on-the-fly, using human chemical and spatial intuition in practical and accessible ways. While humans cannot defeat computers in terms of speed and efficiency for producing results, human perception remains a vital part of drug design and discovery. The ability to be present within a molecular simulation and interact with it holds great promise.

Sophisticated VR hardware is now widely available and relatively not very expensive. Currently, this includes HMDs and controllers. More ‘natural’ (i.e. hand-like) controllers such VR gloves are emerging and allow more tactile exploration of biomolecular structures. It is also possible to harness senses other than just sight, e.g. touch: haptic feedback could be potentially very useful for drug design e.g. feel the energy change and identify the bottleneck to pulling a drug out of a binding site. Other human senses such as sound could be utilized to aid in exploring molecular systems, adding new ways to convey information about a model and its responses, and indeed this is being investigated [139–141].

A particularly interesting emerging frontier is IMD in VR. While IMD is by no means a new concept, the concept of humans interacting with real time MD simulations in a VR is still relatively new. Molecules are inherently dynamic, and molecular motion is important to consider when modeling biomolecules. sSome dynamics important to drug design occur on the order of nanoseconds to microseconds, e.g. loop opening and drug binding, it may be useful to coax the system to explore areas of high energy using IMD-VR that may not otherwise be reached using standard MD simulations, i.e. using IMD-VR to generate rare but important states such as transition states for binding, or to drive conformational changes. While this kind of biased MD is not exclusive to VR (there are many non-VR enhanced sampling techniques available such as metadynamics, umbrella sampling, the string method, etc.), being able to physically reach out and maneuver molecules to find binding modes and pathways for drugs holds immense potential.

Finally, the collaborative nature of interactive VR means that many users can occupy the same virtual space and even interact with the simulated molecules together. This means that scientists across the globe can inhabit the same simulation and observe the same molecular phenomena. This could be useful for sharing new drug candidates or protein structures with colleagues, rapidly and intuitively, with full appreciation of 3-D molecular structure.

Interactive VR is a rapidly developing area. The VR programs for molecular modeling mentioned in this review have been published within the last 6 years, with the majority being published in the last 3 years. VR software and hardware will continue to evolve rapidly. In the next ten years, it is likely that a VR setup will be a staple for drug designers, structural biologists, and medicinal and computational chemists. VR will also allow research teams to collaborate virtually. With the ease of viewing and manipulating 3D static and dynamical molecular properties, the ability to interact MD simulations, and the ability to change structures on the fly, interactive VR will be an increasingly important tool in the armory of computational drug design and development.

Article highlights

• Interactive VR here refers to software where the user can alter the molecular structure of the system in VR, i.e. not just using controllers to interactively change the view of the system by e.g. translating or rotating the object.

• Interactive VR users can interact with physically rigorous molecular dynamics simulations to steer the simulation, by simply reaching out, taking hold of the atoms, and guiding them around the simulation space, e.g. to open or close protein loops, and flexibly dock drugs to protein targets.

• Head mounted displays (HMDs) are the best VR hardware to use for interactive biomolecular modelling and manipulation.

• Interactive VR also allows new ways for drug designers to collaborate virtually, e.g. using the cloud.

This box summarizes key points contained in the article.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Additional information

Funding

This work received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme. It was also supported by the Engineering and Physical Sciences Research Council (EPSRC), The Royal Society and the Axencia Galega de Innovación. RK Walters thanks the Engineering and Physical Sciences Research Council (EPSRC) for a PhD studentship. J Barnoud acknowledges funding from the EPSRC(programme grant EP/P021123/1[HD1]) and financial support from the Agencia Estatal de Investigación (Spain) (REFERENCIA DEL PROYECTO/AEI/CÓDIGO AXUDA), the Xunta de Galicia - Consellería de Cultura, Educación e Universidade (Centro de investigación de Galicia accreditation 2019-2022 ED431G-2019/04, Reference Competitive Group accreditation 2021-2024, CÓDIGO AXUDA) and the European Union (European Regional Development Fund - ERDF)[BJ2] . DR Glowacki acknowledges support from the European Research Council through consolidator grant NANOVR 866559, and also thanks the Axencia Galega de Innovación for funding as an ‘Investigador Distinguido’ through the Oportunius Program. AJ Mulholland acknowledges funding from the EPSRC (grant number EP/M022609/1, CCPBioSim). AJ Mulholland also thanks the ERC for the PREDACTED Advanced Grant (Grant agreement No. 101021207). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.