Cinematic rendering offers entirely new possibilities for studying human anatomy

Medical images constitute a source of information essential for disease diagnosis, treatment and follow-up. In addition, because of its patient-speciﬁc nature, imaging information represents an important component required for advancing patient-tailored precision medicine into clinical practice. By characterizing patient anatomy, physiology and metabolism, medical imaging can enable precise, personalized procedures and predictive, patient-speciﬁc therapy selection and delivery. For this reason it is important to continue to develop means of handling image information. Cinematic rendering, a 3D visualization technology, is capable of producing high-quality photorealistic images from traditional computed tomography (CT) or magnetic resonance (MR) volume data, thus potentially enhancing the ease of viewing anatomical structures and their pathologies. These striking new visualizations could simplify communication between physicians and patients or between radiologists and referring clinicians, help surgeons select the most appropriate surgical strategy, and open up new horizons in medical training.

Medical imaging plays an increasingly important role in moving precision medicine, the science of delivering care tailored to each individual patient, into clinical practice. Efficient clinical decisions and procedures require rapid appreciation of relevant information contained within medical images. Although medical image viewing based on multiplanar reconstruction is still dominant in diagnostic imaging, the significance of 3D visualization of medical data is increasing: the methods allow a much faster understanding of the spatial relationships between anatomical structures and they have the potential to increase the sensitivity and speciﬁcity of medical image interpretation ( ). Drs Klaus Engel and Robert Schneider (Siemens Healthineers) have developed a visualization technology, cinematic rendering (CR), that provides unprecedented photorealistic clinical images by leveraging the physics of light. Refined in collaboration with Professor Franz Fellner (Head of the Central Radiology Institute at Kepler University Hospital, Linz, Austria and extraordinary professor at the Friedrich-Alexander University Erlangen-Nuremberg (FAU)), CR is a physically based volume rendering method that provides hyper-realistic representations of, for example, fractures, organs or the structure of fine blood vessels ( , ) ( Fig. I.4.1 ).

Physically based volume visualization techniques reproduce complex illumination effects in computer-generated images by mimicking the real-world interaction of light with matter. The results are physically plausible images that are often easier for the human brain to interpret, since the brain is trained to interpret the slightest shading cues to reconstruct shape and depth information ( Fig. I.4.2 ). Shading cues are often missing from computer-generated images based on simpler geometric calculations such as ray casting, which produce images without precisely depicted tissue depth and structure and hence less realistic 3D volume visualization ( ) ( Fig. I.4.2 ).

Photorealistic clinical images obtained by leveraging the physics of light

As its name suggests, CR owes its origins to film technology that enables characters to appear entirely lifelike on screen despite having been digitally added to the scenes in which they appear. Although a character is digitally modelled and post-edited into the film, its lifelike appearance is achieved via a technique known as image-based lighting, where a spherical panorama is captured using a reflective sphere. The sphere records the current light environment for subsequent application to image datasets. To understand how these images are rendered, we must turn to the physics of light. Rays of light are made up of particles called photons that interact with their environment. When light encounters matter, it is reflected, bouncing off the surface in various directions: in some places it is absorbed, producing shadows. CR mimics this behaviour.

In the case of CT and MRI images, a physical rendering algorithm which uses a Monte Carlo path tracing method simulates the complex interactions between photons and the scanned images of a patient’s body ( ). Unlike the digital modelling used in cinema, CR not only calculates how light is reflected off the surface of the body but also takes into account how light penetrates tissue and is then scattered in different directions. CR generates photorealistic images by calculating realistic lighting by light transport simulation along hundreds or thousands of photon paths per pixel through the anatomical structure using a stochastic process ( Fig. I.4.3 ). In this way, even complex effects such as ambient occlusion (the effect of how hidden a surface is from the ambient lighting) can be modelled. For example, when the depth of a fracture is fed into the calculations, the deeper the fracture, the less light is able to penetrate, resulting in a range of shadows and a more realistic depiction. Diagnostic 3D medical imaging has not previously leveraged these special characteristics of light ( , , ). The result is an enhanced depiction of fractures, ivory-coloured bones, and clearly defined organs and blood vessels, all of which can be more easily discerned as a result of the inclusion of shadows and the impression of depth ( Fig. I.4.4A ). The process requires vast amounts of computing power: depending on the image resolution, hundreds or even thousands of light paths per pixel must be calculated ( ). The images can be rotated on the screen as desired or zoomed in for greater detail ( Fig. I.4.4B ). Since rendering occurs in the postprocessing stage, the patient is not exposed to additional radiation.

Fig. I.4.3, Volumetric Cornell Box: traditional volume rendering technology (left) versus cinematic rendering (right) . The consideration of light shafts, scattering, caustic reflections, ambient occlusion, colour bleeding and soft shadows enables photorealistic representations.

$Fig. I.4.4, A , Cinematic rendering of a CT, head to pelvis. The patient has multiple fractures of the ribs and spine, as demonstrated in the zoomed view ( B ).$

You're Reading a Preview

Become a Clinical Tree membership for Full access and enjoy Unlimited articles

Become membership

If you are a member. Log in here