Assistance System for Preventing Motion Sickness

Project background

In the KARLI project, solutions were developed to detect, avoid and prevent physical impairments caused by driving dynamics. The prevention of kinetosis contributes to safety in the event of a necessary takeover of the driving task. Fraunhofer IOSB’s camera­based assistant for analyzing poses was equipped with the ability to recognize activ ities in real time. This information was used to implement adaptive interactions in order to initiate preventive user or vehicle measures against kinetosis. 

Motion sickness (kinetosis) is not a new phenomenon, it has so far mainly affected vehicle passengers. Up to two thirds of them are affected by kinetosis. What is new, however, is that this problem also affects drivers who temporarily withdraw from the driving task in an automated vehicle. Especially in SAE level 3 vehicles, in which drivers are allowed to perform various secondary tasks, kinetosis often occurs as a consequence. If drivers suddenly have to intervene again, kinetosis can become a danger: The symptoms – from nausea and dizziness to slower reaction times – sometimes impair driving ability over a period of hours. This makes kinetosis in automated driving not only a comfort issue, but also a safety issue.

Project goal

The idea behind the KARLI-project, which was launched by Fraunhofer and Continental, is to prevent kinetosis before symptoms occur. KARLI is a German acronym for “Artificial Intelligence for Adaptive, Responsive and Level-Conforming Interaction in the Vehicle of the Future” and was co-funded by the then Federal Ministry of Economic Affairs and Climate Action (BMWK). The aim of the project, which involves several partners from the fields of research, industry and services, is to identify hazardous activities for kinetosis and take targeted countermeasures – for example by providing early indications of critical driving routes combined with recommendations for changing sitting posture or activity. The most appropriate time for the proactive instructions should be coordinated with the current activity in order to cause as little disruption as possible and to have the best possible effect.

Project result

Based on the findings from the international survey, Fraunhofer IOSB has developed a detection system for risky activities, based on camera data. The sensors installed in the vehicle – typically already present in semi-automated vehicles – are enhanced by activity recognition based on Artificial Intelligence (AI). This creates an adaptive system that predicts when a certain activity could become dangerous in combination with an upcoming bend, a roundabout or a stop-and-go situation. This information can in turn be integrated into the design of instructions or navigation suggestions or transferred to assistance systems for vehicle control. 

The basis for activity recognition is the AI-supported system already used by Fraunhofer IOSB in several research and development projects, which recognizes 3-D poses in camera data. The activities of the vehicle occupants can be derived from the poses. By analyzing movement patterns and object interactions, the further developed flexible and modular Advanced Occupant Monitoring System can distinguish up to 35 activities, including reading, writing or turning from the front seat to the back seat. The precise and flexible detection of activities in real time with the Advanced Occupant Monitoring System was used in the KARLI project to identify potential risks of kinetosis at an early stage and initiate appropriate countermeasures.