Subsurface Scattering: The Secret to Relevant Medical Imagery is Below the Surface

You’re more transparent than you think.  

You are holding your hand up to a flashlight. As the light shines onto your hand, some of it enters the skin and bounces around inside. This causes the light to scatter in all directions, and escape at points that are a distance away from where they entered, creating a soft, warm glow that appears to come from within the skin.

This effect is called “subsurface scattering,” and simulating this complex phenomenon is crucial when recreating organic materials. The flashlight demonstrates this effect in the extreme—but in its more subtle day-to-day form, subsurface scattering is a critical element in film, video games, and medical training. Subsurface scattering (SSS) serves as a bridge over the “uncanny valley,” and is often a major differentiating factor between “cheap computer-generated images (CGI)” and “the good stuff.” 

Why does subsurface scattering matter to dermatological diagnosis?

In dermatology and other medical fields, the effects of subsurface scattering are crucial to the diagnosis of skin disease, and to monitoring its severity over time. A great deal of aesthetic dermatology is also focused on modifying the behavior of skin below the surface. Diseases and treatments that impact skin layers below the surface will affect the subsurface scattering of light, creating subtle, visual clues recognized by the trained dermatologist. Diseases may impact blood flow, or layers below or above pigment that might only be revealed under certain lighting conditions. Because diseases may cause changes below or above pigment layers, subsurface scattering has a great deal of impact on how diseases reveal themselves on different skin tones.

AAD Blog derm slideshow

Images from Top Derm, Level Ex’s interactive experience for dermatologists.

Where it started: The evolution of realistic video game graphics

Subsurface scattering in early CGI

The technology behind the first CGI films in the 1990’s couldn’t achieve this effect, which is why characters in those films look somehow “wrong” or “artificial.” People describe these characters as looking like they are “made of plastic,” which makes sense, as cheap plastics don’t exhibit subsurface scattering.

“Andy” in the original Toy Story and in Toy Story 4. Photo: Pixar Animation Studios

I wrote my first subsurface scattering “shader” program in 1999 at the University of Illinois, simulating the behavior of light through each layer of skin: Different skin layers absorb and scatter light at different frequencies based on factors like pigment and blood flow. I included this in my application when I applied for my first job at LucasFilm.

In film, we have hours to calculate all of the light physics for each frame, but in video games we must do it at 60 frames-per-second. At Microsoft, my DirectX team invested a great deal of effort into making real-time SSS possible for in-game characters, creating technologies such as “precomputed radiance transfer (PRT),” a crazy mathematical “hack” that can calculate much of the costly SSS light simulations before a game is run. 

Various DirectX precomputed radiance transfer examples provided to game developers. 

Mid-2000s: The video games industry’s race to make more realistic visuals 

The first video games to achieve SSS in real-time were released in the mid-2000’s to great fanfare. The race was on among game developers to adopt these incredibly complex techniques. I discussed this subject during a presentation in 2006 when I took on the daunting task of explaining precomputed radiance transfer to artists. It was the highest-rated talk at Microsoft’s game developer event that year.

SSS Off (Left) vs On (Right) on a video game character.

How it’s going: Advancing clinical education with hyper-realistic medical graphics at Level Ex

At Level Ex, we invest heavily in subsurface scattering research—in creating our own custom, shaders, and algorithms that run on mobile devices, cloud simulators, and VR/AR devices. 

SSS Off (Left) vs On (Right) on a virtual patient with skin of color.

A cube made of virtual skin, one of many tools used to test iterations of Level Ex’s skin rendering technology.

Although skin is the most prominent example of subsurface scattering, this phenomenon is important in many other therapeutic areas. For example, in Gastro Ex, our interactive experience for gastroenterologists and colorectal surgeons, the tissue glows as it is stretched toward the bright light of the colonoscope:

An image from Gastro Ex, our interactive experience for gastroenterologists and colorectal surgeons.

The Bottom Line

Without accurately simulating subsurface scattering, medical imagery risks not only being inaccurate, but also potentially misleading. Accurate simulation helps doctors develop crucial mental models to intuitively understand how a disease (especially skin disease) manifests in the many layers of tissue.

Luckily, strong consumer demand for more realistic characters in video games and film has unlocked massive investment in this problem. The resulting solutions yield results that are indiscernible from reality—solutions that the healthcare industry can leverage to advance the practice of medicine.

Interested in learning more about Level Ex’s technology and how it’s advancing clinicians’ knowledge and improving their medical skills? Contact us.