Cognitive performance decreases with age. This process can affect everyday life and activities to a degree where one can no longer manage their day-to-day activities without assistance. Even people with physical disabilities, such as the blind or the deaf, often need support in everyday life. The problem is that they can only interact with audiovisual technology to a limited extent.

The purpose of the Design Adaptive Ambient Notification Environments (DAAN) project was to implement a technical platform for the systematic support of people whose cognitive performance has decreased or is physically limited.

My role was that of a design researcher. I was responsible for identifying the problems by formulating multiple case studies and conducting a thorough UX research and strategy. I had to come up with metrics, personas, and concepts, and help the team understand the issues at hand. In the project’s later stages, I piloted the UX/UI designer efforts for the Caloo ecosystem, designing and ideating things like the app, the voice assistant, and the ecosystem’s web experience.

Problem

There is a plethora of smart devices on the market that allow you to monitor your health and safety. Interacting with such devices can be overbearing, even more so if you suffer from a cognitive degenerative disease.

Solution

Caloo is a semi-ambient assistant which reflects your healthy past and assists you in your daily tasks. Caloo uses existing smart devices, connects them centrally, and transforms their data into natural notifications and gentle reminders.

Project background

Caloo was the first project of the DAAN (Design Adaptive Ambient Notification) German federal 3P program, which spanned multiple years. DAAN aimed to allow public entities (such as universities and clinics) and private companies to work together on projects and technologies that would enable cognitively disabled individuals to lead better and safer lives. As a media lab, we were heavily involved in developing new ideas and prototypes, working closely with medical professionals and universities.

Our contributions were:

Design research

Traditionally, one would start coming up with hypotheses, ideate, or jump straight into a sprint. However, my intuition told me that it would not work this time. If we were to do that, we wouldn’t be able to grasp the severity of the challenge at hand.

I asked myself, what does it feel like to be cognitively impaired? How do the lives of these individuals differ from mine?

I arranged a meeting with the head of geriatrics at the Bad Cannstatt Hospital in Stuttgart and prepared a user interview that followed strict interview guidelines.

A planned hour-long interview became an entire week of education. I was shocked at the amount of valuable feedback and disappointment I received.

It was emotional seeing people suffer from Alzheimer’s dementia — not being able to remember their name or loved ones. Their detachment from reality. The suicide nets in the stairwell. It left a strong and lasting impression on me.

During my interview week, I gathered a plethora of information and feedback. After documenting the information, I proceeded to analyse it for its severity and impact.

By the end, I could deduce the main issues early-stage dementia patients and their next-of-kin face today when looking for technology to assist them:

My job was to spearhead our research effort. We started delegating tasks and coming up with an initial roadmap. Later it shifted to designing the Caloo Ecosystem.

I started learning more about medical geriatrics and digital assistive living. I read numerous books and papers. To understand the problem on a macro level, I also familiarised myself with the German healthcare system. It was interesting to know how the caregiving fund is structured and how much assistance people with cognitive disabilities can expect.

I created multiple high-level user stories and personas. Below you can find my favourite one, meet Klara:

26% of elderly people in Germany live like Klara.

Using my gathered insights as a foundation, it was to time to ideate. During our brainstorming sessions, we had many great ideas, two of which we wanted to pursue. In this story, I am going to focus on Caloo. The project includes:

• Manda – A virtual semi-ambient assistant.
• Caloo App – An app that helps dementia patients lead a safer and better life.

Cognitively disabled people, especially those who have Alzheimer’s dementia, forget their true self. They forget who they were and, over the years, lose essential motor skills.

My research has shown that the worst thing you can ask a person who has Alzheimer’s is the question: “Do you remember?”

It creates a surge of anxiety and anger. Those affected are physically incapable of remembering certain events or actions. Reminding them of their disability makes things much worse.

Both mildly and severely affected patients have daily routines and activity preferences. Therefore, it is important to offer them avenues that work towards a healthier lifestyle.

According to my findings, the first lifestyle changes are usually initiated by the next of kin. Today, they primarily buy smart devices for their loved ones, sometimes more than three.

I came up with the idea to create an assistant which would use the smart devices inputs and data to transform the insights into semi-ambient audiovisual cues.

I focused on three key user moments:

Audiovisual communication is essential when dealing with cognitively impaired users. My research has shown that the best way to interact with cognitively disabled patients is through multiple non-obtrusive stimuli. The careful interplay between stimuli allows for better recollection and daily activity remembrance accuracy.

I had the idea to make a mandala-style voice assistant named Manda.

In Buddhism, a mandala is a circular or symmetrical figure representing a universe. Mandalas are used to focus attention during meditation and act as a spiritual guidance tool. In Jungian psychology, a mandala is a symbol in a dream, representing the dreamer’s search for completeness and self-unity. It’s an ideal foundation for an assistant, a “spiritual guidance tool” for your daily life.

Together with a teammate, I created the first prototype using Unity.

Now that the direction was set, I focused on the point of purchase. Families tend to buy many smart devices for their loved ones who suffer from dementia. My research has shown that this can be quite confusing and complicated for them.

“Which devices do I need to buy?” “How will it help my father?” “Will it be there for him in case of an emergency?” “How will it improve his daily life?”

All these questions prompted me to create a web-configurator with Manda as its centerpiece. The idea was as follows: With each smart device, and the inputs that it generates, Manda would evolve in real-time. An indicator at the bottom of the page also gives you more details in written form. Overall, it is a playful, fun, and interactive way to show how each device will create a better assistant that will support the family member in the future.

Privacy is a concern. I believe it’s essential to show users how their data is going to impact their lives. If one knows the benefits, the willingness and product acceptance will be considerably higher.

Now that the point of purchase was prototyped, it was time to focus on the Caloo app.

Reflection and Learnings

Prototyping the Caloo app and Manda was a lot of fun. It was both creative and challenging. Coming up with new means of interacting with a user that greatly differ from the ways we are used to (such as mobile notifications) is exciting.

It is crucial to stress the importance of good research. Not only does it offer you a solid foundation on which to build, but it allows you to see things you wouldn’t have seen otherwise. For instance, going to a dementia ward was truly eye-opening.

On reflection, nearly all mobile smart devices have a common flaw: battery life. The further dementia progresses, the harder it is to complete even the most basic tasks. We all forget to charge our phone from time to time; now imagine yourself being physically incapable of doing so.

Forgetting to charge your smart devices is a potential safety risk to all people that require care.

This problem was challenged in the DeepInsight project.