A research team at The Hong Kong Polytechnic University (PolyU) has developed a patient-centric AI Virtual Patient Simulation System, that aims to overcome the limitations of conventional static diagnosis. The system is particularly suitable for cancer and critical care, where disease progression can be complex, treatment options diverse and medical costs high, injecting fresh impetus into the development of precision medicine.

The system creates a continuously updated digital twin model by dynamically integrating multimodal patient data, including genomic data, medical imaging and clinical records, the team said. It was designed not only to track changes in a patient’s condition in real time, but also to predict the potential effectiveness of different cancer treatment options, helping healthcare teams formulate more precise and personalized medical systems.

Other medical AI tools often rely on a single CT scan, genomic report or static clinical data for analysis, making it difficult to gain a comprehensive understanding of dynamic changes in a patient’s condition, according to the team. The system the team developed combines a platform for healthcare professionals with a patient-facing mobile application. It was built to conduct predictive analyses in response to real-time changes in a patient’s condition and simulate the effectiveness of different treatment options.

Further, it provides intelligent support for clinical diagnosis, condition monitoring, and treatment assessment, according to the team. The system’s core strength lies in the close collaboration it enables between healthcare professionals and patients.

The dedicated healthcare platform integrates multimodal data, including genomic data, medical imaging, pathology reports, laboratory test results and clinical records, helping doctors gain a comprehensive overview of a patient’s condition, enhance diagnostic and treatment decision-making, and streamline multidisciplinary consultations and referral processes.

At the same time, the patient-facing mobile application was designed to enable patients to upload medical records, log daily symptoms, and track their health status. Through an encrypted deep feature QR code, medical data can be securely transferred across different clinics, hospitals and devices, enhancing data-sharing efficiency while safeguarding privacy, the team said. With the system, patients can shift from passively receiving treatment to actively participating in their health management, further strengthening doctor-patient collaboration.

To advance the application of this technology in cancer care and treatment decision-making, the research team has introduced a clinical, data-driven, multi-scale AI framework for predicting immunotherapy response in patients with non-small cell lung cancer. The multimodal approach integrates histopathological image features with clinical data, including gene expression profiles and cancer-type text. Named the Visual-Global Relation Fusion Network (ViGNet), the framework incorporates both a multi-scale visual encoder and a gene-driven encoder, enabling AI to analyze image and genomic features that are closely related to cancer treatment response.

In qualitative and quantitative evaluations, ViGNet outperformed baseline approaches in response classification, achieving 82.55% discrimination performance in predicting immunotherapy response.

 “The AI Virtual Patient Simulation System is an innovative and comprehensive platform that integrates diagnosis, monitoring and treatment assessment,’’ said Lawrence Chan, an associate professor of the PolyU Department of Health Technology and Informatics. “In addition to identifying subtle yet crucial pathological connections across multimodal data, the system can also act as a ‘monitoring sentinel,’ alerting healthcare teams when a patient’s biomarkers or symptoms show abnormalities.”

This technology helps to shorten diagnosis and assessment times, Chan added, “supporting healthcare professionals in developing more precise, effective and personalized treatment plans for patients with cancer or other critical illnesses.”