Painless solution

An augmented-reality system designed by Wright State University researchers that puts a “doctor in the house” to help recovering surgery patients manage pain without painkillers was a winner in the Ohio Opioid Technology Challenge.

The technology was developed by Yong Pei, associate professor of computer science and engineering, and David Martineau, medical director of the Hand and Orthopedic Center of Excellence in Kettering Health Networks, along with a group of graduate researchers: Miteshkumar Vasoya, Ashutosh Shivakumar, Rutvik Patel, Yogesh Chavan and Andy McNutt.

The researchers work at Wright State’s SMART Lab, which stands for Sustainable – Mobile – Autonomous – Real Time – Translational. It is located in Wright State’s Joshi Research Center.

The technology was among five winners of the challenge selected from hundreds of ideas submitted by researchers, caregivers, service providers and individuals from Ohio, other states and nine countries.

The $8 million Ohio Opioid Technology Challenge is part of a strategy Ohio is pursuing to fight the deadly epidemic tied to prescription painkillers. The effort, spearheaded by Gov. John Kasich, comes in a state among the hardest hit by the deadly opioid epidemic. There were 4,050 overdose deaths in Ohio in 2016, many linked to heroin and synthetic opioids like fentanyl.

From left: Wright State researchers Miteshkumar Vasoya, Ashutosh Shivakumar and Yong Pei designed an augmented-reality system designed that puts a “doctor in the house” to help recovering surgery patients manage pain without painkillers. (Video by Kris Sproles / photos by Erin Pence)

The Wright State project, in partnership with Kinematechs LLC of Cincinnati, began four years ago as part of investigating the use of artificial intelligence, augmented reality and motion analysis in applied medical research and using technology to effectively deliver health care to patients.

It specifically focuses on patients who undergo major joint surgery such as knee or hip replacements. More than 1 million Americans have such surgery each year.

“This population is particularly at risk of opioid dependency because they depend heavily on the use of strong narcotic medicine,” said Pei.

When the patients are released from the hospital and sent home to recover, they are often alone, unprepared for the pain and lacking medical support, Pei said. That makes using prescription painkillers more than a little tempting.

Pei and his researchers developed a system of cameras, sensors, smartglasses, processors and speakers that would put doctors, nurses or therapists into the patients’ homes in the form of holograms — three-dimensional images formed by light beams.

A key part of the system is the use of the Microsoft HoloLens, a pair of mixed-reality, head-mounted smartglasses that project holograms inside the visor.

The holograms would help patients actively manage their pain by serving as medical advisers, exercise instructors and cheerleaders. The holograms would be images of the caregivers themselves or could be modified to resemble a patient’s family member.

Graduate researchers Ashutosh Shivakumar, Miteshkumar Vasoya and Yong Pei, associate professor of computer science and engineering, developed the augmented-reality technology in Wright State’s SMART (Sustainable – Mobile – Autonomous – Real Time – Translational) Lab.

The system can also be connected to caregivers at the hospital, who instruct the patient in pain-reducing exercise techniques and visually monitor their movements and recovery progress. Patient data is charted and analyzed and can be used to quickly modify the rehabilitation to maximize its effectiveness.

The researchers have developed a prototype, and Pei expects the system to be operational within two years. He says a major benefit of the technology is giving the patients the medical and psychological support they need to avoid the use of painkillers.

“It’s going to be your best friend while you’re at home,” he said. “We realize that this is something that can make a difference. … I think there is a huge potential.”

A native of China, Pei earned a Ph.D. in electrical engineering from Rensselaer Polytechnic Institute in Troy, New York. He joined the faculty at Wright State in 2003.