How 3D Reconstruction of Organs is Beneficial

As stated earlier, 2D images, such as CT scans and x-rays, have primarily been used for pre-surgical planning in procedures dealing with soft tissue-based procedures. However, with 3D technology, surgeons can get a better idea of the anatomical structure of the organ and the location of the organ relative to other soft tissue. With this invaluable insight, surgeons can create a route for the surgery and plan the procedure with more efficiency.

When only relying on x-rays or CT scans, there is a potential for valuable information to be missed during the pre-operative planning process. This could potentially slow down or create issues during the procedure. When using a 3D model of an object, the surgeon can easily rotate and view the object from many different angles. If printed, the object can be held and observed from a realistic perspective, which can give insight into the size of the object. Pre-operative planning using 3D models can lead to better diagnostic accuracy.

Furthermore, this technology enables more collaboration between surgeons. With 3D models, it is possible for teamwork to happen regardless of geography. For example, a surgeon who lives in another country can easily access a 3D model to diagnose a problem and plan an operation. This creates the possibility of experts in certain topics becoming more accessible when their advice would be beneficial for diagnosing or treating a patient.

AR/VR Surgical Solutions

Aside from 3D printing, 3D models could be made for the liver, lungs, or heart. These 3D models could provide the basis for AR/VR reconstruction of the anatomical object. Anatomical knowledge of the operative site is essential for soft tissue-based surgeries. VR and AR offer new ways to view three-dimensional anatomical structures for pre-operative planning.

VR allows the surgeon to evaluate and get familiar with complexity of the surgical area and object in a virtual environment. On the other hand, AR allows 3D objects to be projected onto the patient both before and during surgery. This technology could provide insight that could help the surgeon operate faster with more accuracy than when planning operations using only 3D images.

The Future of 3D Reconstruction of Anatomical Objects

Pre-surgical planning is critical to achieving the best possible surgical result and minimizing any problems during or after the surgery. While there are limits to planning with 2D images, creating 3D models of anatomical objects could result in more efficient pre-operative planning and, therefore, better outcomes for patients. This technology provides the building blocks towards a broader solution of developing AI-assisted surgical tools to provide better overall surgical outcomes.

To learn more, check out the case study.

What is Endoscopy?

An endoscopy, often known as “EGD,” is a minimally invasive technique that takes images of the upper digestive system without requiring complicated and possibly risky surgery.

With endoscopy, doctors can get comprehensive imaging of parts of the small intestine using a long, flexible tube with a small, high-quality camera and light connected to the end. The camera has two lenses that act as the left and right eye, which generates a perception of depth and provides information about how deep a surgeon can go during surgery.

An endoscopy can take biopsies in addition to getting comprehensive pictures of the upper digestive system. A biopsy involves the removal of a tiny sample of tissue. The tissue sample is subsequently submitted to the lab for additional testing to identify the existence of a disease.

Endoscopy may be performed to repair an issue if one is identified. Endoscopic procedures can be used to treat a variety of issues, including opening constricted regions, regulating bleeding, removing abnormal growths or tumors, removing stuck items, and banding any aberrant veins in the esophagus.

Why Does Depth Perception Matter to the Creation of AI Solutions?

Surgeons view pictures through an endoscope or assistive camera in minimally invasive surgery. Because the camera has both left and right eyes, the surgeon has good depth awareness after a few surgeries.

The difficulty is that the AI will need to comprehend depth to automate specific aspects of the procedure.

Therefore, depth perception is essential for AI models.

In minimally invasive operations, surgeons must mentally reconstruct the 3D patient anatomy from a restricted number of visual cues. Insufficient depth signals from the environment may cause surgeons to overestimate spatial depth, compromising patient safety.

Using AI surgical tools can lead to a better knowledge of the spatial connections between different surgical and anatomical objects, as well as ways for the surgical team to minimize risk while allowing human specialists to maintain control.

RediMinds built a model to test out how AI might be used in endoscopy for depth perception. The model showed promising results, with mean average inaccuracy of 3mm across two separate test scenarios – indicating depth perception could be predicted well.

A surgeon can view more perspectives if they have access to 3D structures of a situation. This could allow for the development of more accurate and helpful safety measures.

The Future of AI in Endoscopy

AI has the ability to enhance all aspects of endoscopy, from surgical planning to post-surgery outcomes. It’ll compensate for human mistakes and limitations by improving endoscopic treatments’ accuracy, consistency, and speed.

AI has performed well thus far in endoscopy use cases. However, more research, ethical approval, and new regulations will be required before such AI tools can be used in practice. Nevertheless, AI will be a significant advantage in future years, as it has the potential to revolutionize endoscopy.

To learn more check out the full case study.

How is this Beneficial?

With the application of ML models to surgery and establishment and implementation of situation-specific intervention procedures, the surgeon can take advantage of this window of opportunity to manage patient risks. Predictive models developed in this manner can assist surgeons in identifying high-risk groups, anticipating significant occurrences, and planning treatment measures to enhance patient outcomes. A great case of this in action was performed by RediMinds.

RediMinds worked with the Vattikuti Foundation to create models for predicting intraoperative complications, or in other words, complications before and after surgery, and 30-day morbidity. The models were built using a multi-institutional dataset that was focused on a specific cohort of kidney cancer patients who underwent Robotic Partial Nephrectomy.

Machine learning and AI models performed well in this study in predicting problems during surgery and 30-day complications following surgery. These tools could be applied to real life clinical settings, which would allow the centers to utilize the models to identify patients who are likely to experience a problem before or after surgery, as well as to innovate intervention methods and offer feedback on model performance. This would also allow researchers to track the models’ performance in a clinical context.

Model Development

Here’s a brief rundown of RediMinds’ model deployment process for this study. First, historical data from patients who underwent robot-assisted partial nephrectomy is obtained, including demographic, pre-operative, intra-operative, and follow-up information. The next step, the data engineering process, then includes pre-processing historical data and defining features that will allow machine-learning algorithms to be trained on the data.

Next, model training and model validation models are trained, and the model with the greatest performance on unseen test data is chosen. The model with the highest performance is put in a clinical setting for further evaluation. The final step is the model monitoring, which is the process of comparing model predictions against ground truth to see if the model is operating as expected.

Final Thoughts

Surgeons strive to make their actions and decisions to treat patients under their care as predictable as possible. With the successful application of ML, it is possible to predict complications before and after surgery, which could be consequential to the patient’s surgical outcome.

This approach and methodology may be used in any surgical circumstance across the healthcare spectrum. It is not restricted to Robotic Partial Nephrectomy patients. With adequate data and continued research, the next generation of assistive AI surgical tools can be built utilizing the technique that RediMinds used.

To learn more, check out the case study

This isn’t the only way that our research into AI in healthcare is improving patient outcomes.

Check out our guide to 6 exciting applications of AI in healthcare.

The Challenge of ICU data for ML models 

AI and machine learning (ML) tools models are the future of medical care. These tools allow hospitals and medical facilities to provide better care for patients and research new surgical techniques and medications. However, researching how well tools and models work in the medical environment and training ML models with past data is a vital first step.

The best way to analyze data and perform research like this is by analyzing the data of past patients. As stated earlier, EHRs have tons of patient data, so much that it’s complicated to train a ML model using this data. There are many elements and information collected on patients in the ICU. From the moment they arrive at the hospital, to follow-ups, to surgeries, there are so many pieces of information. This information isn’t even fully structured data. Most of it’s just doctors’ notes, random information, and pictures, and it’s all in different formats.

Training a ML model is a huge challenge when trying to analyze historical data from the ICU. To train models, the data has to be formatted in a specific way. For instance, one doctor at one hospital may label something as one variable, while another doctor may label it as another variable. The variables are usually the same, although the names differ amongst EMRs.

In certain EMRs, Heart Rate may be coded as ‘Heart Rate’, whereas in others it is coded as ‘HR’. This does not prevent model training, but it makes transferring a model from one facility to another difficult. To accomplish the model transfer in this situation, it would be necessary to map new data variables to old data variables names, then conduct data processing and model training processes. Trying to apply a learned model or procedure to several additional hospitals increases the complexity. Hence why interoperability standards like HLv7 and FHIR are created to make data storage more standardized.

Now, multiply that situation by thousands, and now there is a vast amount of data that needs to be cleaned before researchers can use it to train a ML model. Data cleaning is a huge process for data scientists. Researchers spend a lot of their effort gathering and preparing chaotic digital data before using it to build algorithms. Data comes in all different forms. Data scientists attempt to automate as many procedures as possible, but human language is ambiguous and requires data to be cleaned manually.

In fact, some research has indicated that preparing and maintaining data for analysis takes up around 80% of a data scientist’s work. For ICU data, this would mean going through thousands of variables to ensure they are all formatted correctly. However, it’s vital to have clean data because it provides better outcomes and more accurate ML models.

Natural Language Processing as a Solution 

A potential solution to this issue that RediMinds have researched is the application of natural language processing (NLP). NLP analyzes language in text and speech using computer-based approaches. Think of it like this. When we talk to Google Home or Alexa, those devices break down our sentences into tokens. For instance, if you say, “Alexa, light switch on,” the device breaks down the sentence to “light,” “switch,” and “on” and performs an action that it has been trained to do based on these tokens.

Regarding using NLP in the ICU for historical data analysis, NLP models can look for words in EHR notes most commonly used in patients with certain conditions. For example, a patient with a heart issue probably has several words like “cardiac” or “arteries” in their notes. These words can be collected by a ML model, which then can train models on what happens next to this patient.

RediMinds investigated this technique because handpicking or curation of variables takes a long time and may result in the omission of potentially significant data items. The RediMinds team was able to analyze and use data for model training without rejecting large quantities of material by utilizing NLP methods.

Wrapping Up – Why is this Important? 

Using NLP techniques to process both structured (tabular data) and unstructured (Physician’s Notes) cuts down on the time it takes to develop new models for patients. This technique also makes it easier to develop subsequent models because researchers can use the same data. Finally, this technique leverages entire datasets instead of hand-selected variables, which helps eliminate biases of what researchers think is important for the model. With this technique, researchers can tap into valuable insights that may have gone previously unknown due to hand-selecting variables to include in models.

Using NLP also leads to interoperability. With so many different ways to record data, it is nearly impossible to build ML models to make ICU predictions efficiently. However, this technique can help with gathering clean data for building future models.

Based on the first 24 hours in the ICU, ML models can answer questions about patient survival chances, how long patients will be hospitalized, and make several other predictions. If a ML model predicts that a patient will not survive more than a certain amount of hours, then this indicates to doctors and ICU staff that this patient needs to be checked on frequently. This type of AI tool in the ICU has the potential to save lives.

To learn more, check out the case study.