Ultrasound Part 1: Training Your Customers like Astronauts

How Med Device Companies are Using Ultrasound Tech Developed for NASA to Accelerate Training and Adoption

It isn’t just velcro—numerous investments that were originally made to solve problems for the space program have yielded huge benefits here on planet Earth. Read on to find out how ultrasound simulation is charting the same path.

Let’s start with a little background on NASA’s approach to solving challenges related to space health

The Translational Research Institute for Space Health (TRISH) was established by NASA to solve the human challenges associated with deep space exploration— in short, to advance space health for astronauts. TRISH tirelessly pursues and funds high-impact scientific research and technological solutions to help humans stay healthy and thrive on long space flights and expeditions.

As you can imagine, the traditional first step of the engineering process— defining the problem— becomes an order-of-magnitude harder in the context of an activity like space flight. This is why NASA and TRISH work to imagine almost every possible scenario when it comes to defining the problem. When you do that, something interesting happens. New solutions tend to emerge that may not have been considered with a more linear problem-solving approach. In other words, radical problems demand out-of-the-box thinking; and this out-of-the-box thinking begets an incredible range of possible solutions.

Now let’s dive into a real world example.

The Problem: Just-in-Time Ultrasound Training in Deep Space 

A scenario:

Imagine a human spaceflight crew that is nine months into a Mars mission. One of the astronauts suddenly grabs his chest and passes out. In a resource-constrained environment, what can the crew members do? 

There’s a flight surgeon back on the ground in Houston, but at that distance, it can take 40 minutes or more for the communications to make the round trip from Mars to Earth and back. There’s an ultrasound on board, but ultrasound images are difficult to interpret for humans with no medical training—not to mention being in space adds its own complexities to the problem. In this case, microgravity causes the human heart to physically change shape and alters the direction of blood flow. 

Given this challenge, how does an astronaut know whether the ultrasound’s image they are looking at is normal for someone who’s been in space for 9 months? How can we enable astronauts to quickly train themselves on how to perform ultrasound-guided diagnostics and procedures remotely, no matter where they are in the solar system?

A Daunting Set of Requirements for Ultrasound Simulation

After surveying the landscape of ultrasound training simulators available today, the team at TRISH quickly realized that existing commercial ultrasound simulators leave much to be desired. 

Most of these simulators work by taking a slice of a simple 3D model and adding some simple post-processing (noise, blur) to create ultrasound-like images and training experiences. None meet the realism requirements of NASA.

TRISH reached out to Level Ex in 2018 after their assessment was complete. Given our team’s track record of achieving complex computational fluid dynamics on an iPhone using adapted video game technology, their team wondered— was it possible to use a similar approach that would allow us to realistically simulate ultrasound in space, and in real time?

The requirements for this challenge were intimidating to say the least. To start, we knew we were going to need an incredibly realistic ultrasound training simulation. This is to ensure astronauts feel fully immersed and confident in carrying out an ultrasound-guided procedure— and most crucially, to ensure accurate technique and diagnostics can be achieved. 

Specifically, the simulation had to provide:

• Realistic simulation of millions of ultrasound waves bombarding an array of tissues and participating media (in physics, participating media or mediums refer to the material or empty space through which signals, waves or forces pass.)
• Accurate recreation of ultrasound artifacts— i.e. any interfering feature in an ultrasound image that doesn’t accurately represent the examined area (ringdown, acoustic shadowing, etc.)
• Technological support for astronauts to be able to access real patient data, so images can be compared to ultrasound, MRI, or CT scans taken on earth before their space flight
• Real-time, interactive performance (30 frames per second), with no perceivable lag
• Optimized for mobile devices used in space— in this case, a lightweight, low-power tablet 

The Result

To support NASA’s requirements for a completely reimagined ultrasound training experience, Level Ex’s team of video game developers, physicians, and medical experts created the most realistic real-time ultrasound simulator available to date. 

Our simulator being used to scan the jugular vein in an early prototype for TRISH/NASA.

Our ultrasound simulation technology for TRISH:

• Runs at 30 frames per second, enabling better resolution and image accuracy
• Scales across devices—from the low-end mobile phones of today to high-end desktops and Level Ex’s cloud gaming platform, Remote Play™ 
• Features physics-based ultrasound simulation including simulated backscatter, wave propagation, speckle, and realistic interactions with a range of participating media
• Includes a wide range of realistic ultrasound artifacts including ringdown and acoustic shadowing. These artifacts emerge from the simulation as opposed to being added on as a special-case visual effect as is common in today’s ultrasound simulators)
• Can be configured to recreate any transducer parameters and layout— from POCUS (point-of-care ultrasonography) to IVUS (intravascular ultrasound; used for scenarios like evaluating the coronary arteries that supply the heart)
• Is able to generate realistic ultrasound simulation from voxelized patient data. Note: Voxels are to 3D what pixels are to 2D

Thanks to our team’s brilliant work on this important advancement in space health, Level Ex was recognized with a prestigious award at Siggraph, the world’s most influential conference for computer graphics and visual effects. 

You can watch my live demo from the conference here. 

Our simulator is actually validated against phantom tissue analogues—this ensures the simulator generates the same result as an actual transducer:

Ringdown, acoustic shadowing, and other artifacts automatically emerge from the simulation, and are validated against ultrasound phantoms.

You can try a version of it here.

How Our Med Device Clients Are Using NASA-Approved Ultrasound Technology Today

Level Ex maintains the terrestrial rights to its work with NASA, which means that our medical device clients (including Philips, Boston Scientific, and others) are leveraging it to accelerate the adoption of their own ultrasound-based and ultrasound-guided products here on Earth.

This typically involves one (or both) of: 

1. Quickly demonstrating the benefits of ultrasound-based and ultrasound-guided products
   And
2. Training professionals on proper usage of a device across a range of scenarios

understanding that the benefits of a new ultrasound probe often requires “playing with it,” for example patient cases where the device offers the greatest benefit. Phantoms don’t cut it.

In this example, Level Ex’s simulation framework was used to demonstrate the benefits of Philips’ IVUS probe. Here, Philips shows how and why certain lesions are much better visualized with IVUS compared to x-ray fluoroscopy. 

By looking along the simulated IVUS probe, players are able to find a hidden thrombus (blood clot) that would otherwise be invisible under x-ray. Without proper identification via the best-suited devices like this one, blood clots may go on to embolize, causing a stroke. 

You can play through this example by downloading our Cardio Ex app- available on just about every major smartphone.

Other client examples include:

• Demonstrating the benefits of ultrasound-guided procedures and diagnostics
• Demonstrating the capabilities of specific transducers and software

In all cases, realistic ultrasound simulation allows the player to intuitively understand the benefits of the procedure or medical device in the context of relevant patient cases. This ensures the use cases for our client’s various solutions and medical devices truly shine. As an enormous added benefit, end users like ultrasound technicians can play through these experiences remotely without having to go through in-person training regimens or travel to simulation centers to learn how to use the device.

Popular Use Cases for Our Video Game Technology According to Real Med Device Clients

Ultrasound waves are a mysterious and confounding phenomenon to a human brain that has spent its entire life learning how to interpret visible light. Sound can cast a shadow, and it can bounce like light (sort of)—but reacts very differently in different materials (especially fluids and gasses). It can also echo—creating the appearance of objects that aren’t there.

To master ultrasound-guided procedures, medical professionals need to develop an intuitive mental model for ultrasound. Research shows that this kind of “brain training” is best achieved using video game technology and design.

Level Ex’s med device clients are reaping the benefits of this highly engaging, memorable and convenient approach to training thanks to our realistic real-time simulation technology. Medical professionals across the globe can now sharpen their skills and understanding of  medical device usage and best practice techniques across a range of patient cases. 

These skills include:

• Configuring the transducer to achieve the best image quality
• Ultrasound-guided needle placement and navigation
• Diagnostic interpretation
• Use of ultrasound-integrated software (for example: to size a stent, or to enhance an ultrasound image)

An emerging use case we’re now seeing is Just-In-Time training—wherein the clinician trains on a specific procedure or use case right before they need to perform it. This is a crucial scenario for NASA—one that is explored in the next blog post in this series.

In this example, a user is learning to size a stent using Philips’ software recreated inside Cardio Ex.

How it’s deployed:

Level Ex’s cloud-based simulation platform ensures all of the training examples above can be completed independently (e.g., on the user’s phone) or collaboratively (e.g., training sessions hosted onZoom, Microsoft Teams, in-person, or hybrid). Our clients often combine this type of training with their existing training programs—as a prelude to onsite training or as a follow up to keep skills sharp and drive retention of new information. 

Clients often integrate training into their existing single-sign-on (SSO) and learning management systems (LMS).  

Building such groundbreaking tech for NASA and TRISH to advance space health has been an incredible honor for our team. Using these latest ultrasound simulation technologies developed for NASA, our medical device clients accelerate training efforts and the adoption of their products, which ultimately contributes to better outcomes, shorter sales cycles and happier end users. By instilling an intuitive understanding of the benefits of an ultrasound (or ultrasound-guided) device, our clients are improving outcomes and reducing device abandonment thanks to space health-inspired training solutions.

Interested in learning more about Level Ex’s technology? Contact us.