UT Dallas Researchers Lead the First-Ever Trans-Pacific Collaborative Mixed Reality Plant Walk
Dr. B. Prabhakaran, professor of computer science and director of the UT Dallas Multimedia and Computer Graphics Lab, and a team of researchers led the collaboration of the first-ever, trans-pacific collaborative plant walk. The Trans-Pacific Collaborative Mixed Reality Plant Walk involves three different locations (or sites), three different time zones, on two different continents. This collaborative effort included researchers from UT Dallas, Keio University in Tokyo, Japan, and SuperComputing (SC) 2019 in Denver, Colorado. The research done in this project was supported in part by the United States Army Research Office (ARO). The project was commended for its significant contributions to immersion, interactivity, and collaboration.
Dr. Prabhakaran and his fellow researchers evaluated a guided group travel experience involving participation from three geographic sites connected through 100 Gbps optical infrastructure: Dallas, Texas, Tokyo, Japan, and Denver, Colorado. Each site involved one user or participant. At each site, the system involved two Kinects (positioned front and rear of a participant) for content generation, content visualization, interaction detection, and one Oculus Rift to render the plant walk Mixed Reality (MR) environment with all virtual exhibits and “live” models of participants.
An MR tour can be taken independently (known as an unguided travel) or collaboratively (known as a guided group travel). Through an MR tour, a participant (or a group of participants) can virtually explore an area of interest and interact with its virtual objects. In this work, researchers considered a virtual botanical garden use case to educate participants on tree taxonomy.
“This type of collaborative mixed reality systems and applications help in multiple ways. For instance, people with mobility impairments can have virtual experience of a botanical garden from their homes. During this COVID-19 pandemic, people are not able to travel and socialize, leading to social isolation. This system can mitigate such disadvantages by allowing them to have virtual, social interactions,” noted Dr. Prabhakaran.
In this group travel, participants traveled together virtually and visited the tree exhibits where they took a moment to listen and learn new information on the tree they were viewing, just as they would have with a real-world field trip. During this point of the tour, as they traveled virtually, Dr. Prabhakaran and his fellow researchers conformed to a leader-follower strategy, whereby one of the leaders of the virtual group controlled the entire group’s travel. For the virtual exploration, Dr. Prabhakaran and his fellow researchers adopted a gesture-based travel technique. In their experiment, a total of 22 users participated and contributed to the group experience of this trans-pacific, group travel.
“Presence, co-presence, and interaction fidelity are the three pillars that hold realism in a collaborative MR environment. Is Quality of Experience (QoE) a function of Presence, co-presence, and interaction fidelity? This thought inspired us to propose a modified Virtual Reality (mVR) triangle paradigm. We devised and employed the mVR model to evaluate and benchmark the user experience of the trans-pacific, group travel,” noted Shanthi Vellingiri, CE Ph.D. and researcher in the UT Dallas Multimedia and Computer Graphics lab, when analyzing the results of the project.
Dr. Vellingiri noted that at SuperComputing (SC) 2019, the average network delay between sites of Denver and Tokyo was around 125 ms to 135 ms, between Denver and Dallas, the network delay was around 30 ms to 35 ms. She further explained, “The mVR model categorized the user experience of the MR group travel for varying network delay scenarios, including the user experience for a worst-case delay of 173 ms. Overall, the group travel experience was satisfactory. However, we observed intermitted visual rendering, travel, perforated models, and delayed interactions in several trials of the experiment.”
Others involved in this effort included Dr. Andrea Fumagalli of the UT Dallas Open Networking Advanced Research (OpNeAR) Lab, Vania Gi of the UT Dallas Office of Information Technology (OIT), and Drs. Naoaki Yamanaka and Satoru Okamoto of Keio University, Tokyo, Japan.
ABOUT THE UT DALLAS COMPUTER SCIENCE DEPARTMENT
The UT Dallas Computer Science program is one of the largest Computer Science departments in the United States with over 3,315 bachelors-degree students, more than 1,110 master’s students, 165 Ph.D. students, 52 tenure-track faculty members, and 44 full-time senior lecturers, as of Fall 2019. With the University of Texas at Dallas’ unique history of starting as a graduate institution first, the CS Department is built on a legacy of valuing innovative research and providing advanced training for software engineers and computer scientists.
