Virtual Reality (VR) and 3D technologies are widely used in various fields such as medicine, education and bioinformatics. In particular, it has shown great potential for application in biomedical data visualisation. Although existing research has addressed several aspects of this technology, there are still many challenges associated with virtual reality in the field of biomedical data visualisation. This study focuses on exploring the latest applications within the biomedical field and examines how Virtual Reality Learning Environments (VRLE) can provide novel insights and a deeper understanding of tumour biology.

In building the BioVirt Lab platform, we describe in detail the design ideas, development process, experimental design, and data evaluation guidelines of the platform. The development of the BioVirt Lab went through four key stages: preliminary research, data collection and processing, visual modelling and scenario building of the data, and development of the VR application.

1. Introduction

1.1 Tumour biology and microenvironment

With the rapid development of medical science and technology, cancer research has been at the forefront of efforts to find more effective methods of prevention, diagnosis and treatment. According to statistics, millions of people around the world are diagnosed with cancer every year, which makes cancer one of the leading causes of death worldwide. Cancer is not just a health issue; it has a huge impact on economies, societies and families around the globe. To address this worldwide challenge, it has become crucial to understand the biology of tumours and their microenvironment.

A tumour is not an isolated entity; it is intimately connected to its surroundings. Tumour cells interact with their surrounding cells, stroma, blood vessels and immune cells to form a complex "ecosystem". Each part of this system affects tumour progression, aggressiveness and response to therapy[1]. In addition, factors in the tumour microenvironment, such as hypoxia, acidity and immunosuppression, pose challenges to tumour treatment, which is why more complex models are needed to simulate real-life situations.

It is due to this complexity that traditional two-dimensional (2D) cell culture methods are limited in their ability to simulate the real tumour microenvironment. While 2D culture has provided us with a great deal of valuable information in the past, it cannot fully reproduce the three-dimensional structure of a tumour and its interaction with the microenvironment.

1.2 The Rise of Bionic 3D Tumour Modelling

Given these limitations, bionic 3D tumour models have become a focus of research. Compared to 2D cultures, 3D models provide a more realistic platform that allows cells to grow, differentiate and migrate in three dimensions, better mimicking the real tumour microenvironment[2]. These models can simulate not only tumour growth but also drug delivery and metabolism, providing new opportunities for drug screening and personalised therapy.

However, the complexity of such models also brings new challenges to researchers. Observing and interpreting these models requires a high degree of skill and in-depth knowledge in order to construct three-dimensional cell structures from two-dimensional images as well as to accurately depict the dynamic properties of tumour cells. Especially when it comes to analysing cell structure and behaviour through microscopy, researchers are required to have strong spatial thinking skills.

1.3 Application and Prospects of Virtual Reality Technology in Biomedical 3D Model Data Visualisation

As life sciences become more and more dependent on data science, visualisation becomes crucial, especially for data from advanced experimental techniques such as 3D genomics, spatial transcriptomics and macro-genomics[3]. When it comes to VR tools in this field, there are two main types: one is called Non Imaging Examples, and the other is called Imaging Examples. Today, we mainly talk about Non Imaging Examples. The main task of this type of tool is to turn cell data into a 3D form, like a "map" of cells.  Over the past two decades, life science data has grown in volume and complexity, making data analysis a significant bottleneck. With rapid advances in computer science and technology as well as graphics, Virtual Reality (VR) has become an important tool in science, industry, and economics, especially in the "Metaverse Evidence Wall" of biomedical engineering data visualisation[4].

To help researchers better understand 3D tumour models, we propose the use of virtual reality (VR) technology. Virtual reality (VR) provides an immersive environment that allows users to observe and analyse models from all angles[5]. The introduction of this technology not only helps researchers to understand complex 3D structures more intuitively but also provides new ideas for experimental design by simulating experiments such as drug delivery and cell migration.

In addition, virtual reality (VR) technology offers a new way to visualise 3D tissue data in biomedical research. For example, one study used virtual reality (VR) to create a new interface for breast cancer research that can be used to visualise 3D tissue data, thus providing a new approach to cancer research[6]. However, the development and research of virtual reality (VR) platforms for microscopic 3D visualisation of cancer cells has been relatively limited. Indeed, the use of virtual reality (VR) technology is expected to greatly enhance researchers' understanding of 3D models. This is not only because virtual reality (VR) provides a 360-degree viewing perspective, but more importantly, it allows researchers to interact with the models and explore cellular interactions in depth.

2. BioVirt Lab

This research is dedicated to the design and development of an interactive immersive virtual reality learning platform - BioVirt Lab. The development process encompasses crucial sub-phases such as preliminary research, data collection, processing, data visualization modeling and scene modeling, and virtual reality (VR) development. The data used in this project originates from the London University Health and Disease 3D Model Center, while the data processing was a collaborative effort between that center and the Wellcome Trust Human Genetics Center at the University of Oxford. During the modeling phase, besides employing Blender for simulating cell morphology, I also utilized real data to perform data modeling using Blender (Python), engaging in an in-depth exploration of encountered challenges and their corresponding strategies for resolution. Subsequently, I constructed highly realistic three-dimensional tumor models using Unity3D and created animated visualizations depicting the growth of tumors in the surrounding extracellular matrix. Moreover, based on the actual settings of the UCL Bioengineering Laboratory, I reconstructed the laboratory model within Unity. This model was ultimately employed as the teaching and experimental environment for BioVirt Lab, providing simulated experimental scenarios for future research on the effectiveness of virtual reality technology in improving the understanding and cognition of complex three-dimensional tumor microenvironments.

2.2 From 2D to VR

My development process is divided into four core stages: preliminary research, data collection and processing, data visualization modeling and scene construction, and virtual reality (VR) development. All the data used in the project is sourced from the 3D Model Centre for Health and Disease at the University of London. Additionally, the data processing work is carried out collaboratively by the center and the Wellcome Trust Centre for Human Genetics at the University of Oxford. During the modeling stage, I will use Blender to simulate cell morphology and combine real data through Python scripts in Blender for data modeling. Next, I will use Unity3D to create a realistic 3D tumor model and design a visualization animation depicting tumor growth. Furthermore, I have referred to the actual environment of the UCL Bioengineering Lab and created a laboratory model in Unity, which will be integrated into the BioVirt Lab to provide an experimental scenario for studying the effects of virtual reality (VR) technology in enhancing understanding of complex 3D tumor microenvironments.

The pink line shows my development path.

2.3 Preliminary Research

To clarify the requirements, we organised several team discussions at the early stage of the projet, which not only covered the overall framework and objectives of the project but also explored the details in depth to ensure a clear understanding of the project. In addition, to have a more comprehensive understanding of the background and requirements of the bionic 3D tumour model, I have systematically collected and analysed relevant project information. In order to experience and understand the actual application scenarios of the project more realistically, we also conducted experimental field observations, organised Open brush VR Experience Workshops and conducted interviews with experts in the field of bioengineering and actual users. These field experiences and exchanges enabled us to gain a deeper understanding of interdisciplinary knowledge, to concretise and organise the originally abstract concepts and requirements, and to lay a solid foundation for the subsequent development work.

2.3 Data collection and processing

We collected two sets of image data for the bionic 3D tumour model: confocal images and non-confocal images. Confocal images were taken using a 10x or 20X objective lens, which provides finer image details. The non-confocal images, on the other hand, are taken clockwise at a magnification of 2.5× and are intended to capture a macroscopic view of tumour invasion. The image software used to view these data was Fuji.

For further analysis, we divided the data into two groups. the C control group was characterised by a higher activity of cancer cells, meaning that these cells were more likely to proliferate and spread. While the T-treated group had relatively low activity of cancer cells, which could be due to some treatment or other factors. This categorisation helps us to better understand and compare the differences between the different groups.

2.4 3D Bionic Tumour Cell Model Development

The image on the left is a microscopic image of a cell that was processed to obtain a point cloud of the tumour-like body. The data is very noisy, so a thresholding method was used to make the data contain X, Y, and Z coordinates, and a QuPath was used to extract the coordinates of each Z layer, with Z being a "pseudo-depth" based on the Z-index (0- -8), which was multiplied by 100. X and Y are micrometres, so there is a gap of 100 micrometres between each layer.


When it comes to cancer cell data, it is common to focus on the features of the cells, and how these features correlate with factors such as cancer type, grade, and so on. By analysing the CSV files, here are a few possible scenarios for combining features:

In cancer cell data analysis, the focus is often on cell features and their correlation with factors like cancer type and grade. By examining the CSV files, several combinations of features can be considered:

1. Basic Cellular Characteristics: This includes the nucleus and cell area, perimeter, circularity, eccentricity, along with cytoplasm channel means (Ch1, Ch2). This combination helps determine cell shape, size, and cytoplasmic channel expression.
2. Nucleus-Cell Relationship Characteristics: Involves the area and circularity of both the nucleus and the cell, providing insights into their relationship.
3. Nucleus/Cell Area Ratio: This focuses on the ratio between nucleus and cell areas, which may indicate abnormal cancer cell morphology.
4. Multichannel Characterization: This uses mean values from multiple channels (Ch1, Ch2) for the nucleus, cell, and cytoplasm to capture more detailed characteristics.

Due to limitations in data volume (n=371K) and computational resources, I made several adjustments to the model, downscaling the sample size and selecting 2K samples. Ultimately, I found that nucleus-cell relationship features were best suited for the modelling task.

2.5 Simulation of Cell Morphology Modelling

Due to issues with importing the model into Unity, I opted to create a highly realistic 3D tumor model directly in Unity3D. In this step, I focused on using morphological principles to create volume rendering, 3D mesh creation, shaders, and lighting (brightness parameters) to model different cell types in the biomimetic 3D tumor cell model. I actively communicated with the researchers to ensure the accuracy of the visualization.

In Blender, to simulate the fibrous morphology of CD31 endothelial cells, I employed vertex control methods and combined them with vector calculations to precisely control the simulation effects of particles.

To simulate the clustering phenomenon of cancer cells, I used dynamic adhesion simulation techniques in Blender and combined them with physical property settings to ensure the interaction and cohesion of cells.

In Unity3D, I built a three-dimensional tumor model that encompasses over 10,000 individual cell bodies, reflecting the complex structure and characteristics of tumors. To ensure accurate and intuitive data visualization, I tried and applied various visualization techniques, such as advanced shader techniques and mesh deformation techniques, aiming to enhance the realism and detail representation of the model.

2.6 Animation of the growth of the tumour model into the surrounding stroma

The figure below shows that over time, the behaviour of the tumour cells in the orb mimics their natural behaviour in the human body: they cluster, aggregate and invade. When animating the growth of the tumour into the surrounding stroma, I specifically considered the user's perception of colour and intuitive manipulation of space in Virtual Reality (VR).

By observing and analysing the changes and interactions of the stromal cells, we were able to delve into the complexity of the tumour ecosystem. The completed 3D animations can be played back in virtual reality, which not only allows these animation resources to be reused multiple times but also in multiple presentations. Although these animations may not fully reflect the scientific facts, we have endeavoured to ensure transparency in their content and have added rich interactive elements to the virtual reality environment to enhance the user experience.

A. Difficulties:
From the interviews, we learned that it is extremely challenging to continuously observe the movement trajectories of cancer cells through a microscope. Therefore, when creating the cell growth animation, we mainly relied on subject knowledge and laboratory experience for speculation and simulation.

B. Solution:

1. The core of the animation is to show the design of the vascular network in the stromal compartment, simulating how cancer cells "hijack" the body's vascular network in order to obtain the required nutrients. We animated the tumour growth process from day 0 to day 21 to virtually represent the cellular interactions and dynamics in the tumour microenvironment.

2. In order to ensure the accuracy and realism of the growth animation of the tumour model and its surrounding matrix, I introduced a series of adjustable parameters. The purpose of these parameters is to correct potentially misleading visual cues, as well as to avoid visual confusion due to clashes and ghosting between visual elements. To enhance the three-dimensionality and depth of the scene, a variety of visual depth cueing techniques were used:
(1) Visual elements such as colour, blur, relative size, dynamics, background, occlusion and texture were used to enhance the realism of the scene. For example, the dead cancer cells gradually turn grey over time to simulate their changing process in the real environment.
(2) Use the tools "Gather Point", "Center Point", "Controller", "SpawnPos", "TargetPos", "Tunnel", "CubeCollider", and "BallsCore" to accurately control the trajectories and behaviours of the cells to ensure that their movement patterns match the behaviour of real cells.
(3) I avoided the use of cues that could lead to visual conflicts and designed natural transitions and breaks in the animation presentation. This was done to ensure that the animation was presented in a way that was fully consistent with the observations and behaviours of the researchers in the experiments with the bionic 3D tumour model, thus providing a presentation that was both accurate and intuitive.

2.7  BioVirt Lab Scenario Construction

Virtual Reality Learning Environments (VRLEs) offer significant advantages for science education. It allows learners to explore and experience scientific concepts in an intuitive, concrete way, rather than just understanding abstract scientific knowledge by dealing with equations, formulas, or other symbolic representations. For example, students can immerse themselves in a human cell and look around in a three-dimensional perspective to visualise the various organelles and structures within the cell[42]. This three-dimensional visualisation of scientific phenomena not only supports students' conceptual learning but also contributes to the development of their spatial knowledge and skills[43].Notably, the scenario design of BioVirt Lab is based on the real environment of the UCL Bioengineering Laboratory, ensuring its authenticity and practicality.

BioVirt Lab is a comprehensive virtual laboratory, which mainly consists of a teaching area, an experimental manipulation area, a microscopic cell display area and a real data display area. In the microscopic cell display area and the real data display area, students can observe in detail the various components of a tumour cell, such as the nucleus, cytoplasm and stromal cells. These highly detailed 3D models provide an intuitive platform for students to help them gain a deeper understanding of cell structure and function. In addition, this interactive learning experience also helps to enhance students' spatial cognitive skills.

3. References

3.1 Literature: