Goodbye to blind spots: Transparent cockpit thanks to Virtual Reality

Karlsruhe /

Parking assistance systems are widely used. Nevertheless, manoeuvring accidents still happen with cars and trucks, which are mainly due to the vehicle's blind spots. Therefore, scientists at Fraunhofer IOSB research on the "transparent cockpit". A study has shown that this technology can actually help vehicle drivers.

Simulierter Blick aus dem Fahrerhaus eines LKW: Transparentes Cockpit durch Überlagerung von Kameraaufnahme aus Fahrerperspektive und realer Sicht.
Simulated view from the driver's cab of a truck: Transparent cockpit by superimposing a camera shot from the driver's perspective and the real view. The image is the left image of a 180° stereo image.
Testszenario für Nutzerstudie für virtuelles Cockpit. er PKW sollte möglichst nah rückwärts an die Hindernisse herangefahren werden.
Test scenario for the user study by Daniel Secker: The car should be driven backwards as close as possible to the obstacles.
Blick durch die VR-Brille beim Rangieren mit einem LKW
View through the VR-glasses while maneuvering a truck in the study by Daniel Reichert.

In his master thesis at the Fraunhofer Institute for Optronics, Systems Engineering and Image Exploitation IOSB, Daniel Reichert designed, implemented and evaluated a novel vehicle assistance system for maneuvering maneuvers of trucks in cooperation with Rio, a subsidiary of the MAN Group. Subsequently, Daniel Secker developed and tested a real-time system for passenger cars as part of his master thesis. With the help of virtual reality glasses and stereo cameras, it makes the vehicle transparent to a certain extent, so that the driver can see all obstacles directly and three-dimensionally at their real position. Strictly speaking, the system creates an enhanced reality representation, an overlapping of the normal driver's perspective with a virtual view generated by a computer based on the camera shots.

"The goal of this method is to minimize the frequent accidents that occur when maneuvering trucks and cars. Until now, drivers often overlook obstacles or rely too much on existing assistance systems," explains Daniel Secker. Especially for trucks with trailers, it is only possible to capture the relevant environment by using additional mirrors. Even objects directly in front of the vehicle can only be seen indirectly. This represents a potential source of error and increases the reaction time.

Intuitive visualization through camera shots and VR

The transparent cockpit, on the other hand, should allow intuitive visualization. An algorithm recognizes the position of objects in the room on the basis of camera shots and colors them first orange and then red when they come dangerously close.

In a user study, Secker asked 15 test persons to park a car backwards - with and without VR glasses. Cardboard boxes served as obstacles. The result: Drivers can actually judge the distance better through their glasses, but only by highlighting the obstacles in color. The VR glasses alone did not yet produce the desired effect.

However, the virtual reality glasses used are too clumsy for a really practical system, as Secker himself points out. "For this we would need VR or even augmented reality glasses that are suitable for practical use. The driver could wear the latter permanently like normal glasses; only during maneuvers would the virtual display layers be faded in. "Unfortunately, the models available today still have too narrow a field of vision for this," says Secker.

How a transparent cockpit can look is also shown in this video, which was created as part of Daniel Reichert's master's thesis:

Last modified: