From the TV Screen to Real Hospitals: Sci-Fi isn’t Fiction Anymore

By Jennifer Allford

 

The surgeons’ hands move in thin air, exploring a heart only they can see suspended in front of their goggle-clad faces. The doctors examine the hologram to determine the best course of action to fix the heart. Once they decide what to do, their hands move to the operating table where, their fingers wired to a sophisticated surgical robot that’s with the patient in another room, they open the heart, perform the delicate surgery and save the boy.

This scene, from the TV show The Good Doctor, is fiction, but it’s not too far from reality, says Christian Jacob, head of the Department of Computer Science in the Faculty of Science and a professor of biochemistry and molecular biology in the Cumming School of Medicine.

“It’s basically augmented-reality surgery,” he says. “You have a virtual model, displayed like a hologram, and they are finding out with their virtual tools how the surgery could go. We are very close to actually doing that in real life.”

Screens that float in mid-air and display data are being developed now. Augmented-reality glasses are already on the market. “They’re about the size of ski goggles,” Jacob says. “They’re not attached to any other device and they can project almost anything into a virtual space. This is what the guys wore in The Good Doctor.”

Robotic surgery, like that performed in the TV show, was pioneered at the University of Calgary when neurosurgeon Dr. Garnette Sutherland developed the NeuroArm. “All of the technology is available,” says Jacob. “Everything we see in that episode just hasn’t been put together yet. This is not 10 years out or even five years away; it could be done within a year or two.”

 

Eventually, we will get to the point of having all the parts and different levels of the body, so you can drive through the body and go anywhere, just like in The Magic School Bus.

Christian Jacob

Christian Jacob, head of the Department of Computer Science, UCalgary, interacts with LINDSAY

Riley Brandt

Jacob, who works at the intersection of computer science, biology and health sciences, leads the development of the LINDSAY Virtual Human, a computer-generated, 3D virtual person that one can “fly through” to explore human anatomy and physiology.

LINDSAY combines 3D virtual reality and augmented reality with computer game engines to build “immersive worlds” based on the human body. Jacob and his team have already developed augmented-reality versions of different cells of the body and brain. “Eventually, we will get to the point of having all the parts and different levels of the body, so you can drive through the body and go anywhere, just like in The Magic School Bus,” he says.

The iconic animated kids’ show launched in 1994, a few years after Jacob began studying computer science at Friedrich-Alexander Universitat Erlangen-Nürnberg in Germany. He admits that, when he was beginning in the field, he never expected to see technology advance to the point where virtual reality would move into real hospitals and help save lives.

The Magic School Bus in the 1990s and the 1966 medical sci-fi epic, Fantastic Voyage, helped illustrate the concept of moving through the human body in the way the LINDSAY Virtual Human does today. We can thank the Wii computer games console, augmented reality craze Pokémon Go and other 3D developments in the gaming industry for helping researchers advance medical technology.  

“Big companies have put in a lot of resources to make it much easier to create 3D content, (and) computers got faster and they got smaller so we can wear them,” says Jacob. “That really propels the area. We don’t have to have really expensive devices anymore to create content. It’s very accessible, so we can either be in the real world or we can be in this virtual world or we can be in-between, and merge the two.”