Lost Spirit

Gesture-controlled VR flying game & Natural User Interfaces

Xin Tong, Diane Gromala, Alexandra Kiston and Bernhard Riecke

Design Approach.  Lost Spirit is a VR game about flying as a ghost. The goal is to create the most “natural” ways to control navigation in VR through bodily gestures, using one’s arms to “fly.” 

Building on Dr.Gromala’s Virtual Bodies (1991–1999), we explored the novel sensations associated with proprioception as users flew to determine if it may serve as a form of pain distraction OR as a way to recalibrate chronic pain patients’ body image with their body schema. 

Participants in the research study were both ‘neurotypicals’ and chronic pain patients.

Tech. In the game, the player wears an Oculus Rift HMD with headphones, and controls their flying movements using pre-defined arm and body gestures via Kinect’s tracking sensor. 

Numerous alternative movements were explored, and the most effective were tested and compared.

PHASE ONE 

The player can fly or hover above a forest to explore it (or larger landscape), but they cannot reach the clouds or upper atmosphere. Tasks included:

(1) finding and collecting five hidden items in the forest; 

(2) items needed to be handed them over to the final Gate Tree to finish.

PHASE TWO 

After informally demonstrating the game among pain doctors and chronic pain patients at a national pain conference, the team felt that the proprioceptive experience was a promising way to enhance pain distraction. Anecdotal observations of what users deemed “a deeply novel experience” seemed to indicate that the novel sensation of “flying” persisted well beyond the VR sessions — comments we’ve noted in our prior VR research studies.                                         

In 2020, it was being pursued by new team members — experts in neuroscience who have training and permissions to image potential analgesic effects in an fMRI study. Although the fMRI study was stalled by COVID-19, research proposals to finish the study were submitted in July 2024.

Tong, X., Pekcetin, S., Gromala, D., Machuca, F., (2017). “Exploring Body Gestures as Natural User Interface for Flying in a Virtual Reality Game with Kinect,” Paper presented at The Engineering Reality of Virtual Reality 2017 conference, San Francisco, January 2017. Published in Electronic Imaging, The Engineering Reality of Virtual Reality 2017, 1 January 2017. Burlingame, CA: Society for Imaging Science and Technology. https://doi.org/10.2352/ISSN.2470‐1173.2017.3.ERVR‐101 http://www.ingentaconnect.com/contentone/ist/ei/2017/00002017/00000003/art00011

Tong, X., Kiston, A., Salimi, M., Gromala, D., and Reicke, B., (2016). “Exploring Embodied Experience of Flying in a Virtual Reality Game with Kinect” in IEEE VR Mixed-Reality Art Workshop, South Carolina, Mar 2016. Publication + Presentation url: https://www.youtube.com/watch?v=YEWwYfomkYY.