Connecting inner states to a mixed reality sandbox
Garden started within the Potioc research team in Inria Bordeaux, at the end of 2015. This project was a new iteration of the “introspectibles” that were first investigated with Teegi and Tobe (now Echo). This human-size interactive sandbox lets users shape a landscape, which colors and animation evolve not only depending on the topology — piling sand create mountains, digging holes create lakes –, but also depending the physiological signals and inner states, to help the users stay focused on the body — breathing commands the waves crashing on the sandy shores, being relaxed make the forest grow. At the time an additional headset enabled users to immerse themselves in their creation, experiencing it from within, looking up to for the trees, breathing now animating the wind.
Our ambition back then was to investigate up to which point such tool could be use as a support and facilitator for mindfulness, the act of paying a deliberate and non-judgmental attention to the present moment, that had been shown to have a positive impact on a person’s health and subjective well-being. To do so we invited participants versed into meditation (long time practitioners and even a Buddhist lama) as well as medical caregivers (psychologist, psycho-motor therapist).
The feedback and results we gathered indicated that the system was indeed well suited for mindfulness, inducing a calm and mindful state on the user. We also collected many interesting qualitative data, opening up the Garden for new usages. In particular, we realized that such playful multi-modal device could become a tool to alleviate stress or act as mediator to facilitate communication between caregivers and their patients. We then started to envision how the Garden could be used in medical settings and benefit even people with cognitive disorders.
Despite the technical challenges that were hovering above, this is why we started to work, through Ullo, to a portable version of the device, that could be smoothly deployed outside of the lab and used in the field by medical practitioners. It took several iterations to reach this goal, prototypes that were each tested in-situ over the years in partner institutions (nursing home, hospitals, medical and education institutes). Now Garden has finally become an actual product, used across France to help people with diseases ranging from Alzheimer and dementia to autism or ADHD.
Garden was recognized by the scientific community — Honorable Mention Award (top 5% submissions) for its publication at ACM CHI, the leading conference in human-computer interaction; Best Demo Award at IHM 2018, the French counterpart — as well as by the tech industry — CES 2019 Honoree Innovation award Tech For A Better World — or by official representatives — AFNOR certification Testé et Approuvé par les Seniors.
This in only the beginning of the journey though, as we are constantly adding features and imagining new usages for the device — plus of course a packed research agenda. The Garden is currently being investigated as a tool that could be used in education settings, through a pending collaboration with the Nancy-Metz academy and the PErsEUS research team. We are also very interested in pushing further how users could interact with the Garden, connecting new sensors, creating new ways for users to express themselves (from sound synthesis to an “audioscape” matching the land), eventually combining again the Garden with XR devices. Finally we also started to study how features such as the shape of the landscape could bring new information to the table, accounting for users states and helping medical practitioners to better understand their patients (on-going PhD by Camilla Barbini at the CoBTeK lab).
Among the original creators of Garden, check Joan Sol Roo and Renaud Gervais other projects. As about Garden as a product used in the field, more information is available on the Ullo company website.
Associated publications
Joan Sol Roo, Renaud Gervais, Jérémy Frey, Martin Hachet. Inner Garden: Connecting Inner States to a Mixed Reality Sandbox for Mindfulness. CHI ’17 – SIGCHI Conference on Human Factors in Computing System, 2017. 🏆 Best Paper Honorable Mention. ⟨10.1145/3025453.3025743⟩. ⟨hal-01455174⟩ PDF
Joan Sol Roo, Renaud Gervais, Jérmy Frey, Martin Hachet, Martin. Introspectibles: Tangible Interaction to Foster Introspection. CHI ’16 Workshop – Computing and Mental Health, 2016. ⟨hal-01455174⟩. PDF