By combining their creativity with powerful computing tools, University of Miami graduate students Maria Aguilar and Manouj Govindaraju crafted an extended reality (XR) application to welcome incoming freshman. Now, campus newcomers will be able put on a Magic Leap 1 set and “walk” through the campus, learn about academics, athletics and other topics, and listen to UM students talking about their fondest memories.

“Today’s Generation Z students can get bored with conventional orientation programs and campus tours, so we wanted to engage them through an exciting, immersive experience,” said Manouj, who graduated with Aguilar in May with a Master of Fine Arts (MFA) degree.

For Maria, creating an interactive XR application was a “thrilling” way to use powerful computing and data tools to work. “It will help new students make a deep connection with the University, inspire the new generation, and seed their ‘Canes love.”

Launched by UM Student Affairs and supported by IDSC, “The U Experience” combines real-world and virtual environments in an interactive XR application that goes beyond virtual reality (digital only) and augmented reality, which adds a digital dimension to real-world video.

“This is a phenomenal multidisciplinary project that brings together students, faculty and research staff,” said Nick Tsinoremas, Founding Director, IDSC, and Vice Provost for Research Computing and Data. “It demonstrates that that XR coupled with artificial intelligence and powerful computing represent how we will all be interacting, learning and creating knowledge in a hybrid physical and digital world.”

Maria and Manouj looked at the main concerns of Generation Z students and their families about college life, and designed and developed the XR application in keeping with UM initiatives for student orientation and retention, according to Prof. Kim Grinfeder, Director of the School of Communication Interactive Media Program. “From the offset, we wanted to provide a collaborative, forward-thinking project that allowed our students to focus on what’s next in computing,” he added. “The ability to translate knowledge into a digital product is one of our priorities and it’s great to see it come together in such an innovative package.”

Phase one of “The U Experience” was tested in the Magic Leap Lab in the Richter Library last August. “That gave us a great opportunity to try the first version of the app with users, learn from them, and come up with this new version,” said Maria. “The support we received from IDSC and our advisors from the Interactive Media Program was amazing.”

IDSC provided technical support including access to the University’s supercomputer. “The team used a LIDAR (Light Detection and Ranging) laser system to measure distances across the University, and provided us with the generated 3D mesh, which we were able to directly plug into the application,” said Manouj. “The scale of the project would not have been the same, if not for the IDSC’s computing resources.”

Maria noted that the benefits of XR include increased content understanding, better grasp of spatial structures, long-term memory retention, and increased student motivation, and that XR applications can significantly change the way information is delivered to college students.

“This project provides opportunities for future development, such as adding more layers of information, exploring new interactions with data visualizations, and testing the experience in other platforms,” she said. “It also opens up opportunities for evaluation and comparison with traditional programs for freshmen orientation.”

Both Maria and Manouj felt the XR project was a powerful learning experience, helping them develop their data science skills, while contributing to the future of the University. “When designing an extended reality app, there seem to be a million possibilities,” Maria said. “Your artboard is not just a rectangle, it’s a 360° space and it takes some time to start imagining a whole new experience in 3D. But we created a new version of the app that gives the user a feeling of being in a new environment.”

Manouj plans to become an XR applications developer, “Understanding the target users is vital when developing these experiences,” he said. ”I will definitely be using supercomputers in my future.

Maria believes effective use of data is essential for any field in academics or industry. “We need to keep the research, design, and development of products, services and applications based on data,” she said. “I’m looking forward to seeing what’s coming next.”

TOOLS •Magic Leap 1 Creator Edition •Unity 2019.2 •Maya 2020.1 •Blender 2.8 •Adobe Illustrator •Adobe Photoshop •Adobe Dimension •Figma

Paper  |  Final Presentation  | Website

Fine Arts Students Draw on IDSC Resources to Develop Immersive XR Experience for Incoming UM Class-by Richard Westlund