Recognition systems and image exploitation

Recognition systems are systems and procedures that extract higher-level information and knowledge about entities in the real world from incoming data or assist humans in doing so.

The main focus of IOSB’s activities lies on researching and developing recognition systems based on imaging sensors. We employ imaging sensors covering the visual as well as the IR spectral ranges, which are processed as single images or video image sequences. The real-time aspect and process optimization for lightweight hardware and application-specific solutions play a major role in our work. Our recognition methods are primarily deep learning  approaches, which are adapted or developed specifically to fit the task. We train and evaluate our models on our own, extensive deep learning server infrastructure. While we have a wide range of annotated datasets already available for training, we have also established an efficient process for curating and annotating additional training data using several tools and workflows developed specifically for this process. Our work with recognition systems covers are broad range of applications, some of which are illustrated below.

In the field of Airborne Image Exploitation, novel Wide Area Motion Imagery (WAMI) sensors have been developed, which allow a wide area monitoring of several square kilometers, because they have a high ground coverage and a high level of detail (Wide Area Motion Imagery (WAMI Analysis)). At Fraunhofer IOSB, methods for the automatic analysis of WAMI data are developed, which allow for the detection and tracking of persons and vehicles (vehicle detection in aerial photographs). Essential steps in this recognition system are the detection of motion and the detection of objects, which are realized with deep learning methods.

Furthermore, methods for object detection and vehicle classification are realized for other  air- and ground-based optronic sensors. For sea-based sensors, methods for ship detection and recognition are available.

Another major field of work is the detection, recognition and activity recognition of persons. This applies to a range of application domains from aerial image exploitation to the application in camera networks. Major tasks in this type of recognition system are:

 The detection and defense of illegal drones is a major challenge, because potential safety concerns and resulting costs (e.g. in case of airport closures). An initial detection can be carried out with different long range sensors, such as radar or radio detection. For the subsequent differentiation of the drones from decoy objects like birds, flying objects, etc. and for further analysis the IOSB applies video sensor technology. For this purpose, detection and tracking methods for drone tracking with a spotter camera and deep learning methods for the distinction of clutter and recognition of drone types are developed (UAV detection and recognition).

The detection of temporal changes between several overflights with video sensor technology represents a substantial information compression, since only the changed objects needs be investigated. One application example is the inspection of building infrastructure for changes, such as corrosion effects or missing parts. Due to changes in perspective, lighting conditions, and irrelevant movements like vegetation, many changes occur between the images of the different overflights, which are of no actual relevance. Therefore the semantic classification of the objects with deep learning methods is an essential step in the procedures for airborne change detection to identify relevant changes. Corresponding process chains have been realized and are currently being tested.

Education and training of the image exploitation specialists serve to ensure the competence to act and contribute to an effective and efficient use of detection and image exploitation systems. For this purpose we develop intelligent, adaptive learning systems including interactive animations, serious games and gamification methods.

In mobile robotics, recognition systems are required for the detection of the environment. This allows for the identification of drivable paths even off the road in unstructured terrain and the avoidance of obstacles. In addition to LiDAR sensors, which provide a very accurate 3D reconstruction of the environment, camera systems complement the perceptual sensor technology. Here, the additional use of camera systems enables a denser depth estimation as well as the coloration of the LiDAR point clouds and thus an improved mapping and obstacle detection. The AI-based combination of two- and three-dimensional image information allows an exact capture of the environment for navigation as well as the working space of mobile systems.