Abstract
Modern virtual reality (VR) devices record six-degree-of-freedom kinematic data with high spatial and temporal resolution and display high-resolution stereoscopic three-dimensional graphics. These capabilities make VR a powerful tool for many types of behavioural research, including studies of sensorimotor, perceptual and cognitive functions. Here we introduce Ouvrai, an open-source solution that facilitates the design and execution of remote VR studies, capitalizing on the surge in VR headset ownership. This tool allows researchers to develop sophisticated experiments using cutting-edge web technologies such as WebXR to enable browser-based VR, without compromising on experimental design. Ouvrai’s features include easy installation, intuitive JavaScript templates, a component library managing front- and backend processes and a streamlined workflow. It integrates with Firebase, Prolific and Amazon Mechanical Turk and provides data processing utilities for analysis. Unlike other tools, Ouvrai remains free, with researchers managing their web hosting and cloud database via personal Firebase accounts. Ouvrai is not limited to VR studies; researchers can also develop and run desktop or touchscreen studies using the same streamlined workflow. Through three distinct motor learning experiments, we confirm Ouvrai’s efficiency and viability for conducting remote VR studies.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Raw data in JSON format, preprocessed data in XLSX format and Python analysis code are available on GitHub for experiment 1, experiment 2 and experiment 3. Analysis code relies on the Python data analysis utilities that come with the Ouvrai source code on GitHub.
Code availability
The production version of the study code as well as Python analysis code for experiments 1–3 is available in Jupyter notebooks at the following links: experiment 1, experiment 2 and experiment 3. The complete source code of Ouvrai is available at https://www.github.com/EvanCesanek/Ouvrai. The Ouvrai GitHub repository will be updated as new features are developed. Previous releases can be accessed via the commit history of the main branch.
References
Tarr, M. J. & Warren, W. H. Virtual reality in behavioral neuroscience and beyond. Nat. Neurosci. 5, 1089–1092 (2002).
Bohil, C. J., Alicea, B. & Biocca, F. A. Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12, 752–762 (2011).
Tieri, G., Morone, G., Paolucci, S. & Iosa, M. Virtual reality in cognitive and motor rehabilitation: facts, fiction and fallacies. Expert Rev. Med. Devices 15, 107–117 (2018).
Johnson, B. P., Dayan, E., Censor, N. & Cohen, L. G. Crowdsourcing in cognitive and systems neuroscience. Neuroscientist 28, 425–437 (2022).
Cesanek, E., Zhang, Z., Ingram, J. N., Wolpert, D. M. & Flanagan, J. R. Motor memories of object dynamics are categorically organized. eLife 10, e71627 (2021).
Zhang, Z., Cesanek, E., Ingram, J. N., Flanagan, J. R. & Wolpert, D. M. Object weight can be rapidly predicted, with low cognitive load, by exploiting learned associations between the weights and locations of objects. J. Neurophysiol. 129, 285–297 (2023).
Cesanek, E., Flanagan, J. R. & Wolpert, D. M. Memory, perceptual, and motor costs affect the strength of categorical encoding during motor learning of object properties. Sci. Rep. 13, 8619 (2023).
Tsay, J. S., Lee, A., Ivry, R. B. & Avraham, G. Moving outside the lab: the viability of conducting sensorimotor learning studies online. Preprint at https://arxiv.org/abs/2107.13408 (2021).
Tsay, J. S., Haith, A. M., Ivry, R. B. & Kim, H. E. Interactions between sensory prediction error and task error during implicit motor learning. PLoS Comput. Biol. 18, e1010005 (2022).
Tsay, J. et al. Large-scale citizen science reveals predictors of sensorimotor adaption. Nat. Hum. Behav. 8, 510–525 (2024).
Listman, J. B., Tsay, J. S., Kim, H. E., Mackey, W. E. & Heeger, D. J. Long-term motor learning in the ‘wild’ with high volume video game data. Front. Hum. Neurosci. 15, 777779 (2021).
Kim, O. A., Forrence, A. D. & McDougle, S. D. Motor learning without movement. Proc. Natl Acad. Sci. USA 119, e2204379119 (2022).
Bönstrup, M., Iturrate, I., Hebart, M. N., Censor, N. & Cohen, L. G. Mechanisms of offline motor learning at a microscale of seconds in large-scale crowdsourced data. NPJ Sci. Learn. 5, 7 (2020).
Parsons, T. D. Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 9, 660 (2015).
Boland, M. How many VR headsets did meta sell in Q4? AR Insider https://arinsider.co/2023/02/06/how-many-vr-headsets-did-meta-sell-in-q4/ (2023).
Heath, A. This is Meta’s AR/VR hardware roadmap through 2027. The Verge https://www.theverge.com/2023/2/28/23619730/meta-vr-oculus-ar-glasses-smartwatch-plans (2023).
ARtillery Briefs, episode 62: VR usage & consume attitudes ARtillery Intelligence https://youtu.be/unN6nDhX3jo (2022).
VR usage & consumer attitudes, wave VI ARtillery Intelligence https://web.archive.org/web/20230208171407/https://artilleryiq.com/reports/vr-usage-consumer-attitudes-wave-vi/ (2022).
Evans, J., Tsaneva-Atanasova, K. & Buckingham, G. Using immersive virtual reality to remotely examine performance differences between dominant and non-dominant hands. Virtual Real. 27, 2211–2226 (2023).
Jones, B., Goregaokar, M. & Cabanier, R. WebXR Device API W3C https://www.w3.org/TR/webxr/ (2023).
Firebase pricing. https://firebase.google.com/pricing (Google, 2024).
Peer, E., Rothschild, D. & Gordon, A. Behavioral Lab 3.0: towards the next generation of online behavioral research. Preprint at PsyArXiv https://osf.io/preprints/psyarxiv/buzwn (2023).
Peer, E., Rothschild, D., Gordon, A., Evernden, Z. & Damer, E. Data quality of platforms and panels for online behavioral research. Behav. Res. Methods 54, 1643–1662 (2022).
Warburton, M., Mon-Williams, M., Mushtaq, F. & Morehead, J. Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems. Behav. Res. Methods 55, 3658–3678 (2022).
Smith, M. A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).
McDougle, S. D., Bond, K. M. & Taylor, J. A. Explicit and implicit processes constitute the fast and slow processes of sensorimotor learning. J. Neurosci. 35, 9568–9579 (2015).
Heald, J. B., Ingram, J. N., Flanagan, J. R. & Wolpert, D. M. Multiple motor memories are learned to control different points on a tool. Nat. Hum. Behav. 2, 300–311 (2018).
Russo, M. et al. Intercepting virtual balls approaching under different gravity conditions: evidence for spatial prediction. J. Neurophysiol. 118, 2421–2434 (2017).
Gonzalez-Franco, M., Cohn, B., Ofek, E., Burin, D. & Maselli, A. The self-avatar follower effect in virtual reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) 18–25 (IEEE, 2020).
Maselli, A., De Pasquale, P., Lacquaniti, F. & d’Avella, A. Interception of virtual throws reveals predictive skills based on the visual processing of throwing kinematics. iScience. 25, 105212 (2022).
Raghavan, P., Santello, M., Gordon, A. M. & Krakauer, J. W. Compensatory motor control after stroke: an alternative joint strategy for object-dependent shaping of hand posture. J. Neurophysiol. 103, 3034–3043 (2010).
Klein, L. K., Maiello, G., Paulun, V. C. & Fleming, R. W. Predicting precision grip grasp locations on three-dimensional objects. PLoS Comput. Biol. 16, e1008081 (2020).
Ariani, G. & Diedrichsen, J. Sequence learning is driven by improvements in motor planning. J. Neurophysiol. 121, 2088–2100 (2019).
Ingram, J. N., Körding, K. P., Howard, I. S. & Wolpert, D. M. The statistics of natural hand movements. Exp. Brain Res. 188, 223–236 (2008).
Ranganathan, R., Adewuyi, A. & Mussa-Ivaldi, F. A. Learning to be lazy: exploiting redundancy in a novel task to minimize movement-related effort. J. Neurosci. 33, 2754–2760 (2013).
Kieliba, P., Clode, D., Maimon-Mor, R. O. & Makin, T. R. Robotic hand augmentation drives changes in neural body representation. Sci. Robot. 6, eabd7935 (2021).
Cannon, A. R., Wilson, C., Goregaokar, M. & Smith, T. F. W3C immersive web community and working groups face to face 2nd day (Seattle). W3C https://www.w3.org/2020/02/06-immersive-web-minutes.html#item11 (2020).
Eye tracking #79. GitHub https://github.com/immersive-web/proposals/issues/79 (2022).
Cabanier, R. WebXR expression tracking – level 1. GitHub https://cabanier.github.io/webxr-face-tracking-1/ (2022).
Unity – Manual: XR. Unity Technologies https://docs.unity3d.com/Manual/XR.html (2023).
Project Flowerbed: a WebXR case study. Meta https://developer.oculus.com/blog/project-flowerbed-a-webxr-case-study/ (2023).
Brookes, J., Warburton, M., Alghadier, M., Mon-Williams, M. & Mushtaq, F. Studying human behavior with virtual reality: the Unity Experiment Framework. Behav. Res. Methods 52, 455–463 (2020).
Bebko, A. O. & Troje, N. F. bmlTUX: design and control of experiments in virtual reality and beyond. i-Perception 11, 4 (2020).
Schuetz, I., Karimpur, H. & Fiehler, K. vexptoolbox: a software toolbox for human behavior studies using the Vizard virtual reality platform. Behav. Res. Methods 55, 570–582 (2023).
Canning, C. G. et al. Virtual reality in research and rehabilitation of gait and balance in Parkinson disease. Nat. Rev. Neurol. 16, 409–425 (2020).
Nuara, A. et al. Telerehabilitation in response to constrained physical distance: an opportunity to rethink neurorehabilitative routines. J. Neurol. 269, 627–638 (2022).
Heald, J. B., Lengyel, M. & Wolpert, D. M. Contextual inference underlies the learning of sensorimotor repertoires. Nature 600, 489–493 (2021).
Brashers-Krug, T., Shadmehr, R. & Bizzi, E. Consolidation in human motor memory. Nature 382, 252–255 (1996).
Caithness, G. et al. Failure to consolidate the consolidation theory of learning for sensorimotor adaptation tasks. J. Neurosci. 24, 8662–8671 (2004).
Gandolfo, F., Mussa-Ivaldi, F. A. & Bizzi, E. Motor learning by field approximation. Proc. Natl Acad. Sci. USA 93, 3843–3846 (1996).
Karniel, A. & Mussa-Ivaldi, F. A. Does the motor control system use multiple models and context switching to cope with a variable environment? Exp. Brain Res. 143, 520–524 (2002).
Krakauer, J. W., Ghilardi, M. F. & Ghez, C. Independent learning of internal models for kinematic and dynamic control of reaching. Nat. Neurosci. 2, 1026–1031 (1999).
Nozaki, D., Kurtzer, I. & Scott, S. H. Limited transfer of learning between unimanual and bimanual skills within the same limb. Nat. Neurosci. 9, 1364–1366 (2006).
Howard, I. S., Ingram, J. N. & Wolpert, D. M. Context-dependent partitioning of motor learning in bimanual movements. J. Neurophysiol. 104, 2082–2091 (2010).
Howard, I. S., Ingram, J. N. & Wolpert, D. M. Separate representations of dynamics in rhythmic and discrete movements: evidence from motor learning. J. Neurophysiol. 105, 1722–1731 (2011).
Howard, I. S., Wolpert, D. M. & Franklin, D. W. The effect of contextual cues on the encoding of motor memories. J. Neurophysiol. 109, 2632–2644 (2013).
Sheahan, H. R., Franklin, D. W. & Wolpert, D. M. Motor planning, not execution, separates motor memories. Neuron 92, 773–779 (2016).
Alhussein, L. & Smith, M. A. Motor planning under uncertainty. eLife 10, e67019 (2021).
Sheahan, H. R., Ingram, J. N., Žalalytė, G. M. & Wolpert, D. M. Imagery of movements immediately following performance allows learning of motor skills that interfere. Sci. Rep. 8, 14330 (2018).
Ghahramani, Z. & Wolpert, D. M. Modular decomposition in visuomotor learning. Nature 386, 392–395 (1997).
Witney, A. G. & Wolpert, D. M. Spatial representation of predictive motor learning. J. Neurophysiol. 89, 1837–1843 (2003).
Ingram, J. N., Howard, I. S., Flanagan, J. R. & Wolpert, D. M. Multiple grasp-specific representations of tool dynamics mediate skillful manipulation. Curr. Biol. 20, 618–623 (2010).
Mattar, A. A. G. & Ostry, D. J. Modifiability of generalization in dynamics learning. J. Neurophysiol. 98, 3321–3329 (2007).
Krakauer, J. W., Pine, Z. M., Ghilardi, M. F. & Ghez, C. Learning of visuomotor transformations for vectorial planning of reaching trajectories. J. Neurosci. 20, 8916–8924 (2000).
Vetter, P., Goodbody, S. J. & Wolpert, D. M. Evidence for an eye-centered spherical representation of the visuomotor map. J. Neurophysiol. 81, 935–939 (1999).
Shadmehr, R. Generalization as a behavioral window to the neural mechanisms of learning internal models. Hum. Mov. Sci. 23, 543–568 (2004).
Acknowledgements
This work was supported by the National Institutes of Health (R01NS117699 to D.M.W.) and the Air Force Office of Scientific Research under award (FA9550-22-1-0337 to D.M.W.). We thank Z. Zhang and A. Löffler for helpful discussions and testing of Ouvrai. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
E.C., J.N.I. and D.M.W. conceived Ouvrai as a software toolbox for developing crowdsourced experiments on motor control and learning. E.C. created and developed Ouvrai, with assistance from S.S. and J.N.I. All authors conceived of the design of experiments 1–3. E.C. implemented, conducted and analysed experiments 1–3. All authors provided feedback on the presentation of the results, wrote and revised the manuscript.
Corresponding author
Ethics declarations
Competing interests
D.M.W. is a consultant to CTRL-Labs Inc., in the Reality Labs Division of Meta. This entity did not support or influence this work. The authors declare no other competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Gavin Buckingham, Marta Russo, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Figs. 1–2, Firebase Realtime Database and FSM.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cesanek, E., Shivkumar, S., Ingram, J.N. et al. Ouvrai opens access to remote virtual reality studies of human behavioural neuroscience. Nat Hum Behav (2024). https://doi.org/10.1038/s41562-024-01834-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41562-024-01834-7