Virtual Reality – Frost Institute for Data Science & Computing

Ophthalmology Training in the UMVerse

Ophthalmologist Giselle Ricur, M.D., has been promoting telemedicine for decades, but it wasn’t until COVID hit in early 2020 that she started making major headway.

“The pandemic came, and we accomplished more in three months than we had in 30 years.”

“The pandemic came, and we accomplished more in three months than we had in 30 years,” said Ricur. “The pandemic tore down barriers we had encountered before, including government regulations and the lack of insurance reimbursement. I helped many colleagues around the world get their telehealth clinics in place.”

Now, as the executive director of virtual care at the Bascom Palmer Eye Institute, Ricur has an even bigger goal—creating a Bascom Palmer digital twin that will allow physicians, medical students, and others the ability to access immersive educational experiences almost anywhere.

The Value of Simulation

Teaching medical procedures can be a catch 22. Students must train on patients to gain real world experience; however, lacking that experience, they can make mistakes. Virtual simulations through digital learning tools can help solve this problem, allowing students to practice in realistic, low-risk settings.

“Simulation plays an important role because it provides us a risk-free environment to perform complicated procedures,” said Ricur. “It allows people to make mistakes until they master a technique.”

The digital twin project will take the
science of simulation to the next level.

That process is ongoing, as instructors can continually increase the procedure’s difficulty, helping students become adept at performing complex procedures, and addressing rare complications. One example is a very rare complication of cataract surgery called an expulsive hemorrhage. “Until a surgeon has done hundreds of these, it’s unlikely they will ever encounter the issue,” said Ricur. “But what happens if they do? Experience in the simulator can help them make the right, split-second decisions.”

Digitally Replicating a World-Leading Facility

The digital twin project will take the science of simulation to the next level, providing the full breadth of Bascom Palmer’s educational resources via virtual reality headsets or computer monitors. The program will also help UM internationalize its academic offerings.

Ricur is working with Kim Grinfeder, who directs IDSC’s Creative Technologies research focus, and others in the University’s School of Communication, as well as colleagues at the School of Architecture to create the twin, which will be part of the planned UMVerse.

“I’m starting with a blueprint, and then I’m going to start building,” said Ricur. “We’ve had architecture students scan the different areas in Bascom Palmer, such as surgical rooms, so we can create familiar training scenarios for physicians, residents, or medical students, to help them feel comfortable in the digital environment.”

The simulation will include digital equipment, to replicate examinations and procedures. In addition, the digital twin will be equipped with haptic simulators, allowing users to experience pressure, while they practice procedures. Eventually, the technology will help physicians provide remote care.

“My vision is that anything a student needs to train
will be available in the UMVerse.”

“My vision is that anything a student needs to train will be available in the UMVerse,” said Ricur. “Once they’ve validated themselves, they can go into the real world and work with patients.”

Story by Josh Baxt

Mangrove City VR Experience

“Mangrove City“, a VR experience geared towards educating middle schoolers on the crucial role mangrove forests play in protecting coastal communities and mitigating climate change, took home the XR prize for the Educate category at the AWE 2023 Awards. This collaborative project was submitted as part of the “XR Challenge: Fight Climate Change.”

“Students from across campus came together to contribute to this
project and the judges recognized its depth and potential.”

IDSC Creative Technologies Director Kim Grinfeder, (also Department Chair and Associate Professor of the School of Communication’s Interactive Media program and the U’s XR Initiative) says “Immersive Media is interdisciplinary by nature and this project shows how we are breaking down some of the curricular barriers at the University to accomplish something like this. Students from across campus came together to contribute to this project and the judges recognized its depth and potential.”

Prof. Grinfeder and students Kumani Riley and Ashay Dave, along with School of Communication + IDSC XR Developer Bryson Rudolph traveled to Santa Clara, California to present Mangrove City to the judges and the audience. Reflecting on the experience, Bryson commented “We were honored to have presented side-by-side with such talented teams from across the globe. It was truly inspiring to see how others are utilizing XR in unique and innovative ways to address difficult real-world issues. The presentations filled us with hope for the continued fight against climate change!”

Ashay, who worked on making the XR experience realistic with sound effects, shared his take on attending the event in person. “The Augmented World Expo is the premier convention in Extended Reality. I learned more about the industry in four days than I possibly could’ve in the last four months. It was an honor meeting the companies at the forefront of this industry, and the sheer number of talented professionals in the field.”

Reflection on her experience, Kumani shared “It was a pleasure working with my team to create a solution to a global issue, climate change, through immersive technology. It showed me that we can transcend the physical boundaries and create immersive experiences that inspire action, amplify awareness, and accelerate the global shift towards a greener, brighter future. Additionally, the event provided valuable insights and inspiration, encouraging me to further explore and contribute to the XR ecosystem. Overall, it was a profound learning experience that showed me the transformative power of XR, especially to drive positive change in our world. One of my favorite parts of the attending the conference was testing out the world’s first Augmented Reality laptop, Spacetop by Sightful.”

From 230 concept submissions to 150 MVP entries, to 90 teams who delivered, the running was whittled down to just 11 finalists. See Kim, Kumani, Bryson, and Ashay—pictured at left from left to right—accept the award at the award presentation at the 01:07:48 mark on YouTube.

VR Enhances Maternal Mental Health

Virtual reality (VR) headsets have mostly been popularized by children—and some adults—playing immersive video games. But a team of researchers at the University of Miami Miller School of Medicine led by Azizi Seixas the Principal Investigator of the study, Director of the U’s Media and Innovation Lab (“The MIL”), and Director of Population Health Informatics at the University’s Institute for Data Science & Computing (IDSC), is now testing whether VR can help reduce the overwhelming stress that pregnancy, childbirth, and postpartum can have on people.

The study will use a program called Nurture VR, which is a maternal mental health and educational program designed by the company BehaVR to support expectant moms through the third trimester, labor and delivery experience, and postpartum phases. Nurture VR uses a VR headset to educate and immerse mothers in calm settings like green spaces, beaches, and hillside villas, and guides them through mindfulness meditations, trains them to better deal with anxiety, and provides activities like breathing exercises to give them a mental break.

NurtureVR Hoag x BehaVR Peri-Natal Program from BehaVR, LLC on Vimeo.

Judite Blanc, a Research Assistant Professor of Psychiatry and Behavioral Sciences at the Miller School who co-leads the study with Seixas, said mental health is critically important for mothers going through the psychological and emotional stresses of having a baby. She said the nation is facing a “maternal health crisis,” and her hope is that easing the mental health burden facing new mothers will not only help them, but also their newborns.

“Your mental health during the pregnancy period, and as well as the postpartum period, is linked with your overall well-being, with how you are going to initiate relationships with your children, as well as the quality of the care that you’re providing,” Blanc said. “This is what we’re tackling.”

“Ten to twenty percent of all women experience perinatal depression, but those rates double for people of color.”

The problem is even more pronounced for Black and Hispanic mothers who often deal with higher levels of different types of stress depression and medical complications. Partly due to systemic racism, those mothers don’t have as much financial support to hire people to help and don’t have equal access to universal healthcare coverage. In 2020, the maternal mortality rate for Black women of 55.3 per 100,000 was nearly three times that of white women, according to data from the U.S. Centers for Disease Control and Prevention. A separate study in 2020 found that 10 to 20% of all women experience perinatal depression, but those rates double for people of color, leading to “wear and tear” on one’s body, according to the American Hospital Association.

“It’s so heart breaking because America, which is the richest country in the world, has statistics in terms of maternal health and maternal mortality for women of color that are comparable to those of a low-income country,” Blanc said.

Using VR technology to help ease maternal mental health is just one part of the broader goal of using new technologies to improve patient outcomes spearheaded by Seixas. He said their bedrock principles for The Media and Innovation Lab are to develop solutions that are innovative and equitable, meaning they are constantly seeking out breakthrough technologies while ensuring that people from all walks of life can take advantage of them. “We want these solutions to be cool, useful, and impactful,” he said.

The study, which started recruiting participants in December 2022, will include 25 pregnant people and 25 who recently gave birth. They will each receive a VR headset manufactured by Pico for BehaVR and will be required to use them over a 12-week period. Each participant will also receive a $100 gift card as compensation for their time. Blanc and her team will then assess the condition of each participant to see what effect the VR experience had on them.

Blanc, an advocate for universal healthcare coverage and access to free childcare, said their ultimate goal is to make the VR experience for expecting and postpartum mothers available to all, a monumental challenge given the high cost of the devices, the need for a high-speed internet connection at home, and the fact that most insurance companies still don’t cover VR technology or other emerging prescriptive digital therapies. But Seixas said recent political developments have shown that Blanc’s dream of a VR experience for every mother is possible.

Seixas said politicians have started making major investments in mental health as multiple crises have hit the U.S. The former President, Donald Trump, signed into law a sweeping response to the national opioid epidemic that included $1 billion in funding for mental health initiatives. President Joe Biden signed another law that provided $330 million for youth mental health. And Vice President Kamala Harris started a maternal health initiative that culminated in more than $1 billion in postpartum insurance coverage, included in the Build Back Better Act of 2021.

Insurance companies are also starting to cover some prescriptive digital therapies. A bill introduced in Congress this year would even require Medicaid and Medicare to start approving those novel technologies. “Our study really is consistent with a larger policy zeitgeist,” Seixas said. “We don’t think we’re a needle in a haystack, but rather, just executing against a policy that’s been laid out for us. We just need more people to do it to have a larger reverberation so people know these are the options that are available.”

Blanc and Seixas hope the study can contribute to a better understanding of the benefits of VR because after testing it herself, Blanc said, she saw its potential. “As a Black mother, I really wish I had something like that when I was pregnant and after I had my daughter,” she said.

Story by Alan Gomez

ype: Virtual Reality