DeepInsight

DeepInsight is a novel technology that enables the ambient monitoring and assistance of cognitively impaired patients. It drastically improves their daily life by allowing them to live in their own home longer and more safely.

As the lead project manager and inventor, I piloted the development of DeepInsight™. I coordinated with engineers, scientists, designers, medical professionals, and patent lawyers, from devising project / product strategy and delegating tasks to planning sprints and tracking milestones.

I conducted the patent research, formulated multiple patent drafts, and worked with patent lawyers on the WIPO application.

I also owned all features from conception to design (visual and interaction) to prototype-level implementation. These included the product’s machine learning and IoT architecture as well as its UX/UI.

Problem

In developed countries, the population above 65 years of age is rapidly increasing, and so are cognitive degenerative diseases such as Alzheimer’s dementia. By 2025, the caregiving system will reach its maximum capacity. There is an urgent need for the decentralisation of our current health system.

Solution

DeepInsight is a non-invasive ambient assistive living technology which monitors users’ home activities and assists them in case of emergency or caregiving needs. The system relies on only two core sensors. It is affordable and highly scalable.

Project Backround

DeepInsight is the third and final project of the DAAN (Design Adaptive Ambient Notification) German federal 3P program, which spanned multiple years. DAAN aimed to allow public entities (such as universities and clinics) and private companies to work together on projects and technologies that would enable cognitively disabled individuals to lead better and safer lives. As a media lab, we were heavily involved in developing new ideas and prototypes, working closely with medical professionals and universities.

Our contributions were:

In addition to our contributions, I conducted numerous UX research studies and gained many valuable insights into healthcare, dementia, caregiving, and geriatrics.

Design Process

Having worked for multiple years in UX Design and R&D during Caloo and Kopernikus, I had the privilege to continuously collaborate with multiple stakeholders and experts in medicine, engineering, behavioural science, psychology, and research.

It was a humbling and unique experience. But it wasn’t without its flaws. I noticed the following issues, which could partially be attributed to the 3P nature of the project.

My key learnings occurred in the following areas:

I concluded that to create a truly usable, valuable, and enjoyable product, one would have to start with the user experience and work backwards to the technology. Thus, the first step towards the development of DeepInsight was made.

I took the role of a project lead and started the process of developing a novel system. However, there was a challenge that was set by the federal program (DAAN): I had to come up with technology in any shape or form. Balancing design research while scouting for new technologies, not knowing if and how they could be implemented, was truly challenging and unique.

I adjusted the double diamond to fit the project’s needs.

First, I allocated the UX research section of the process to myself. I already conduct multiple UX research studies. The main goal in this project would be to correctly synthesise the results, extract the most informative insights, and define new KPIs.

Second, I replaced traditional design ideation methods with an engineering technique called the theory of the resolution of invention-related tasks (TRIZ), or the theory of inventive problem solving (TIPS). TIPS follows an algorithmic approach to the invention of new systems. Its core idea starts with the concept of a “perfect product” and then, using 40 principles of technical evaluation, derives a reality-based solution. More on that later.

After synthesising my research and liaising with medical experts, I came up with the following problem statement for the DeepInsight project:

“How might we build an ambient system that could assist and monitor dementia patients at home, without them having to interact with anything?”

I then defined the following KPIs and metrics:

I listed a few smart devices and created guidelines on how to measure stimuli levels qualitatively. After giving my colleagues a short seminar, I asked them to evaluate them based on those guidelines.

Analysing the metrics using traditional UX methods such as by 2×2 matrices would not suffice. The dimensionality did not cover all the stimuli. To overcome this, I created a 3D data visualisation. This turned out to be too complex to visualise on keynotes, and a simple spider chart did the trick. Each corner was coded with one sense. The webs represented the stimulation level. The findings were then stored in an Excel table.

To make everything more interactive and to better visualise the results, I transferred the data to Plotly.

An example can be seen below:

Using the TIPS methodology, I came up with multiple “perfect product” statements.

These statements are technical contradictions. The process dictates that closely overcoming these technical contradictions using the 40 principles of technical evaluation and extensive patent/literature review would directly result in novel inventions.

After TIPS practice rounds with the team, it was time to start ideating. We listed the principles that needed to be used and proceeded to the research phase.

Here you can see how I organised this phase:

Multiple ideation sprints were made, yet we could not solve the scalability problem. It was quite challenging to find a solution or technology which would allow DeepInsight to scale fast.

Then one night, I had a revelation. I remembered a technology called nonintrusive load monitoring (NILM). Simply put, it gives insight into a household’s power usage to a degree where one can classify the watched TV shows.

One problem was that NILM cannot analyse water usage, which is essential if we want to know someone’s urine control, bathing, and showering activities. NILM works in conjunction with modern smart meters. The good news is that Germany plans on making smart meters obligatory in every household by 2025. This solves the scalability issue.

To overcome the issue, I came up with the following idea:

Pair a modified NILM smart meter with a fully digital radar sensor smart water meter (SWM). The SWM is easily mountable on any central water pipe and can monitor the water consumption in the millilitre range. The sensors’ (power + water) data output would then be processed using a novel AI to classify home activities.

To better classify activities, I conceptualised a multi-layer AI architecture (which later became part of my patent).

I liaised with an AI expert who very much liked the idea of separating each classification step, and I proposed a classification structure. Below, you can find the advantages of a multi-layer approach:

DeepInsights technology is still in its infancy. Before completing my journey I would love to share a few prototypes and features.

Reflection and Learnings

Inventing a system using user-centred design and engineering principles is truly a fascinating process. With each iteration, you learn something new, and the system evolves by overcoming the constraints nad challenges posed at the time. Patenting the technology was also a insightful experience that taught me a lot.

DeepInsight tackled shortcomings and solved many issues presented by today’s technologies. However, the HMI and technological aspect of the system needs further research. How will users interact with an invisible system? What does the UX of a Zero-UI look like? Can micro-rader technology be used to aggregate better movement data?

Answering these questions is currently part of my MRes (Master of Research) thesis, which I am pursuing at the Royal College of Arts.

I am looking forward to sharing my findings soon. Stay tuned 😀