KARLI - AI for interaction in the vehicle of the future

Competence of Fraunhofer IOSB

In KARLI, Fraunhofer IOSB is developing the Advanced Occupant Monitoring System for the adaptive design of interactions in the vehicle interior. For this purpose, new data from road trips with changing automation levels are collected and refined. Data collection is performed in a Wizard-of-Oz vehicle in cooperation with Fraunhofer IAO. In this process, instructed and freely observed secondary activities on long car journeys are detected by Fraunhofer IOSB AI methods. The Fraunhofer IOSB Drag&Sense framework is optimized in the project in a simple plug-and-play hardware for flexible use in real vehicles. For the first time, we are working with new cameras and lighting adapted to the future automotive drag&senseve series standard, making our solutions compatible with our customers' requirements.

Short description of the project

The goal of “KARLI” (acronym of the German project title "Künstliche Intelligenz für Adaptive, Responsive und Levelkonforme Interaktion im Fahrzeug der Zukunft") is to develop an adaptive, responsive and level-compliant interaction between humans and vehicles of the future and to use AI methods to develop new safety-relevant assistance systems for automated vehicles. The AI know-how is driven by a consortium of strong partners from research and industry. The consortium management is shared between Continental (responsible for administrative management) and Fraunhofer IOSB (responsible for scientific and technical management).

Project goals

For the development of an adaptive, responsive and level-compliant interaction in the vehicles of the future, customer-relevant AI functions are being developed in KARLI that capture driver states and shape interactions for different automation levels. These AI functions are developed in KARLI from empirical and synthetically generated data. The data will be collected and used in KARLI in such a way that the project results are scalable to Big Data from production vehicles available in the future. KARLI provides the specifications and requirements for the vehicle architecture and sensor technology of the future so that upcoming interior sensors collect optimal Big Data for driver-vehicle models and AI interactions.

Project results

The target applications of KARLI are aligned with the overall goal of “level-compliant driver behavior”. The current driving situation, including the automation level, results in concrete requirements for the driver’s state, his behavior and the predictable ability to act. The recording of the driving situation and the driver’s state thus allows an actual-goal comparison and a targeted approach in the dialog between man and machine. Thus, on the one hand, mode and situation awareness on the part of the driver is created and the readiness for transition is promoted, and on the other hand, the trajectory planning and the automation level are adapted to the driver’s state. KARLI promotes driving with the highest possible level of automation, which has a positive effect on efficiency and safety. At the same time, motion sickness, misuse and abuse of the automation function are detected and avoided.