Gesture-based error marking for quality assurance

Quality assurance

In manufacturing, quality assurance is an integral part of meeting set quality standards and customer expectations. Inspection processes are usually closely embedded in the IT infrastructure. Every defect found is entered and documented digitally and often even in parallel using pen and paper. Innovative technological solutions increasingly focus on data exchange, the embedding of testing in production processes and the optimisation of the respective testing steps. Less focus, however, is placed on the human-machine interface itself, with which an inspector enters defect markings into the system himself. Here, people still frequently rely on mouse and keyboard input or documentation via small handhelds and PDAs to fill in input masks for quality assurance accordingly.

Camera-based gesture recognition for intuitive & fast input

The IOSB is developing methods for camera-based person detection and in this context also for pointing gesture recognition. The worker can indicate defect locations directly on the component using intuitive pointing gestures. The worker and the component are localised with the help of a camera, the body pose of the worker is analysed and the outstretched arm is recognised as a pointing gesture. The displayed position on the component can thus be interpreted as defect locations and transferred directly to a three-dimensional model of the component, e.g. the CAD model.

Instead of time-consuming and even error-prone input at peripheral devices, documentation is accelerated, efficient and intuitive. Additional walking distances, which occurred, for example, due to the previous input on a PC, are completely eliminated. Inspectors can thus turn their attention to their actual competence and task without interruption. Error locations no longer have to be transferred from memory to the input mask, which also minimises the likelihood of transmission errors.


Interactive Analysis and Diagnosis Department of Fraunhofer IOSB

You want to know more about our projects and products? Then visit the page of our department.