Project co–financed by REACT-EU to mitigate the effects of the pandemic crisis
Project Title: AI-improved organ on chip cultivation for personalised medicine (AImOOC)
Funding: European Regional Development Fund (ERDF), Measure 1.1.1.1 “Support for applied research”
Project No.: 1.1.1.1/21/A/079
Period: 1 January 2022 – 30 November 2023
Project costs: 500 000,00 EUR
Project implementer: Institute of Electronics and Computer Science
Cooperation partner: Latvian Biomedical Research and Study Centre
Cooperation partner: SIA „Cellboxlab”
Principle Investigator BMC: Dr. biol. A. Ābols
Project summary:
This project focuses on the development of a machine learning algorithm to improve of patient-derived cell culturing in organ on chip devices. Such algorithm development would enable to adopt more widespread use patient-derived material for organ on chip devices, thus allowing scientists in academia and industry to derive more representative model systems. Therefore, the aim of the project is to apply machine learning (ML) algorithms on microfluidics based on bright field microscopy, TEER (Trans Epithelial Electric Resistance) and O2 biosensor data in real time to cultivate different cell cultures (including those obtained from patient samples) on OOC platform. In order to achieve this aim, we have defined the following objectives: (1) organ on chip cell culture data generation, (2) bright field microscopy system development for organ on chip monitoring in real time, (3) machine learning based computer vision algorithm development to process generated data for microfluidics and finally (4) validation of developed algorithm on organ on chip devices by using cells derived from patient samples. The main outcomes of the project are: (1) data in the form of images and sensor read out from lung and gut cell culturing using various flow parameters, (2) development of the moving stage and chip imaging system for use in culture chamber, (3) a machine learning-based system for automating cell culturing and finally (4) patient derived cell culturing in organ on chip systems controlled by the developed machine learning algorithm.
Information published 03.01.2022.
Progress of the project:
1 January 2022 – 31 March 2022
During this reporting period, we have developed decision tree for data classification and produced first imaging data of successful and unsuccessful OOC cultivation data for developing AI models for supervising OOC cultivation. We identified possible approaches to the generation of synthetic data for training AI models. Due to the nature of the real-world data, the most promising approach is to generate synthetic data by means of generating simple geometric shapes and subsequently deforming them. We are currently conducting a survey of literature on that topic. We have investigated integrated objective/camera units for integration in the instrument from various providers with particular focus on evaluation of image quality, digital zoom capabilities and lighting conditions. We have started working on defining the procurement specification for XYZ gantry with a suitable XY step for continuous channel imaging and Z-step for successful autofocus on the aforementioned imaging units. Additionally, during this period, we engaged in public dissemination of project topic in student council of Riga Technical University organised online interview in Spiikiizi studio, titled “What if?”.
Information published 31.03.2022.
Progress of the project:
1 April 2022 – 30 June 2022
During this reporting period, we have generated additionally 230 pictures of both lung and gut on chip models by applying both stable and primary cell lines. For each picture information such as model ID, cell type, seeding density, time of image, decision (good, bad, acceptable), artefacts were prepared. EDI investigated state-of-the-art approaches in literature to the generation of synthetic images of biological cells by deforming simple geometric shapes. Furthermore, EDI investigated the use of generative adversarial networks (GANs) for simulation-to-real transfer, which is needed to render synthetic images more realistic. CellboxLabs conducted several hours interview with potential end users in industry about OOC real time microscopy option in combination with AI to confirm the necessity of such system. Additionally, CellboxLabs made purchases for microscopes to conduct tests with them. They made a purchase for the parts of the XYZ table and currently are working on a design for the XYZ table that would allow us to make an XYZ motion system with motion and small position errors.
Information published 30.06.2022.
Progress of the project:
1 July 2022 – 30 September 2022
During this reporting period, LBMC have generated additionally 250 pictures of both lung and gut on chip models by applying both stable and primary cell lines. For each picture information such as model ID, cell type, seeding density, time of image, decision (good, bad, acceptable), artefacts were prepared EDI researched deep neural network model architectures to find the best fit for the AimOOC task. We also looked for models pre-trained on medical images. A classification model was trained on the first data received from the partners, concluding tha tadditional data is needed for a good result. Therefore, options for data ugmentation that will allow for synthetic multiplication of training examples were also researched and summarized. As part of the project, a contract was concluded and the design of a precision XYZ table was carried out. The construction of the XYZ table was started.
Information published 30.09.2022.
Progress of the project:
1 October 2022 – 31 December 2022
EDI researched methods of improving and multiplying training data. We studied methods included in the various Python libraries and frameworks, choosing and testing the ones best suited to the project images. EDI continued work on the generation of synthetic medical data, looking into the use of generative adversarial networks and novel diffusion models. A data augmentation algorithm was prepared.
During this reporting period, CellboxLabs has produced additional organs on chip devices, while LBMC has generated additional pictures of lung cancer on chip models by applying both stable and primary cell lines. For each picture information such as model ID, cell type, seeding density, time of image, decision (good, bad, acceptable), artefacts were prepared. Cellbox Labs developed necessary firmware to control stage and take photos.
Finally, project supervisors participated in Radio Radio broadcast – The known in the unknown Preclinical studies – organs on a chip and laboratory mice, where they explained to the general public in popular scientific language about technology they are developing.
Information published 30.12.2022.
Progress of the project:
1 January 2023 – 31 March 2023
After finishing work on the data augmentation method algorithm and the development of feature extraction algorithms, EDI continues work on the classification models, dividing the cell images according to time and cell lines. Models are trained according to the decision tree prepared in the earlier stages of the project. To improve model accuracy, we are currently working on two main tasks:
1. images are trained by dividing them into smaller units – quadrilaterals without losing image details and increasing the number of images;
2. work is being done on generating synthetic images with the help of Stable Diffusion to increase the amount of data and, thus, the accuracy of the classifier.
During this period Cellbox Labs have produced 32 additional organ on a chip devices and started to work on Task 4.1 Firmware integration with algorithm, alongside continued work on TEER and oxygen sensors. To accelerate the initial adoption of the algorithm-driven flow-rate adjustments in the channels, a dedicated user interface for manually inputting the desired flow rate will be made. The AI-algorithm developed by EDI is currently utilising flow-rate data, as opposed to shear-force data in the image tagging, subsequently to keep in line with the syntax, flow-rate data will be used.
While LBMC was able to produce around 900 additional microscope pictures using lung on a chip, lung cancer on a chip, and gut on a chip model by visualisation system developed in WP2. Moreover, bright field (BF) pictures and Hoechst staining pictures were also produced and overlapped with BF pictures to aid the machine learning process.
Furthermore, CellboxLabs took part in two seminars organized by LBMC and Riga Technical University. The first seminar was on the role of biomedical research in the knowledge economy and biotech start-up success stories, while the second seminar was focused on the transition from university to industry. During these seminars, CellboxLabs presented their project ideas and direction to a wider scientific and industry audience.
Information published 31.03.2023.
Progress of the project:
1 April 2023 – 30 June 2023
During this time additional 1601 pictures were generated by cultivating gut on a chip, lung on a chip and lung cancer on a chip from stable and primary cells. All pictures were classified by established criteria. EDI worked on improving the accuracy of image classifiers. Various deep neural network architectures were trained using both data supplemented with classic data augmentation methods, and images synthesized with diffusion models. Upon receiving new data from partners, EDI also created new datasets for training, continuously increasing the number of real images used. The trained models were validated on real images, and the results of these experiments were described in the conference paper “Synthetic Image Generation With a Fine-Tuned Latent Diffusion Model for Organ on Chip Cell Image Classification”, which was accepted at the SPA 2023: Signal Processing – Algorithms, Architectures, Arrangements, and Applications conference. To enhance synthetic data, EDI researched additional image generation approaches that still rely on diffusion models, but synthesize images not from random noise, but by modifying existing real images. Working on integrating the trained models with the prototype developed in the project, EDI studied the parameters of the embedded computing platforms to arrive at a version usable in the prototype. Additionally, project topic and results were reported by oral presentation in LU PSK The 3rd international scientific conference “Quality of Health Care and Social Welfare” and MPS world summit 2023 by poster presentation. First tests of OOC growing by implementing established algorithm were started.
Information published 30.06.2023.
Progress of the project:
1 July 2023 – 30 September 2023
During this period, we produced additional 1857 bright field imaging pictures at different time points and quality for algorithm from lung cancer, normal lung and endothelial cell lines.
EDI worked on optimizing the classification algorithm by applying several image generation approaches at the data augmentation stage of the training. Such techniques as Stable Diffusion-based inpainting, LoRA training, and image interpolation were applied to augment the dataset of real images. Validation of these approaches was started.
EDI and LBMC presented the goals and current results of AimOOC project at Scientist’s Night 2023 event on September 29.
CellboxLabs CEO has an interview about technology in company in journal Forbes, CTO visited Rigas Technical University Engineering school, were he explained our technology to pupils and Arturs Abols provided an interview to startin.lv to inform industry and journal Medicus Bonus to inform medical doctors about organ on a chip technology and application within this project.
Information published 02.10.2023.
Progress of the project:
1 October 2023 – 30 November 2023
During the project, we successfully captured over 4,000 bright field images from both successful and unsuccessful experiments involving organ-on-a-chip (OOC) models derived from six distinct cell lines at various time points. These images have been made publicly available in a well-known repository, enhancing the accessibility of our data. Furthermore, our team has submitted a comprehensive article detailing our data collection methodologies and findings.
In addition to our imaging achievements, we have developed a real-time bright field imaging system specifically designed for OOC applications. This innovative system was implemented effectively throughout the project’s duration. Moreover, we created a machine learning algorithm tailored to analyze the images we acquired, as well as synthetically generated images. The development and potential applications of this algorithm are elaborated in a separate article that we have submitted.
A significant milestone was reached when we tested this algorithm and the associated automated decision-making processes on patient-derived iPSC lung-on-a-chip and lung cancer-on-a-chip models. The outcomes of these tests have led to the insightful conclusion that while the model based on stable cell lines is robust, it requires further refinement using data derived from patient models.
The culmination of our project’s achievements was showcased at three international conferences, where our team presented the results and methodologies. Furthermore, we prepared and submitted two research articles, offering a deeper dive into our findings. The project’s final report, including detailed accounts of all tasks undertaken and their outcomes, has also been compiled and submitted, marking the successful completion of this ambitious project.
Information published 30.11.2023.