Artificial intelligence in remote sensing

Search for objects and compress data with AI

For searching and classifying objects in high-resolution satellite images and for compressing hyperspectral data.

Objects detected with AI with class assignment.
Objects detected with AI with class assignment.
Double object detection due to overlapping image areas (base, box NMS) and solution to the problem using ARM NMS.
Visualization of the many channels of a hyperspectral image
Visualization of the many channels of a hyperspectral image

Keeping an overview with AI

AI applications are already widely used in many areas of everyday life. Appropriately trained networks write texts, design images and recognize structures in multidimensional spaces that often exceed human imagination.
Classic tasks in the evaluation of large (volumes of) aerial and satellite image data include searching, finding, classifying and counting various objects. Here, too, the human eye reaches its limits. If the evaluator looks at the entire image, smaller objects are quickly overlooked. If he examines enlarged image sections, it is difficult to maintain an overview of the entire image. This is why automated methods of digital image processing are increasingly being used for such tasks. Whether ships in ports, airplanes at airports or cars in parking lots: modern object detectors support the evaluation of remote sensing data in real time with the help of increasingly powerful hardware components. For this purpose, the large-format image is divided into tiles with overlapping areas and evaluated. Until now, however, this has often led to duplicate object detections in the overlapping areas.

For this reason, Fraunhofer IOSB developed an AI based on the ARM-NMS approach (Area Rescoring Mask - Non Maximum Suppression). Two-stage object detectors first create suggestions for detections and place so-called bounding boxes around the objects. The suggestions are then sorted out using the non-maximum suppression method. The problem with rectangular bounding boxes is that their shape usually does not match the outline of the objects to be detected. The Area Rescoring Mask method is therefore used to adapt the shapes of the bounding boxes to the shape of the object to be detected. The AI has also been trained to reliably evaluate scenes with widely varying object densities, which posed problems for previous approaches.

Remote sensing system with compressed data.
Overview of the 1D-Convolutional Autoencoder architecture.

Save data volume with AI

Conventional RGB sensors (e.g. cell phone cameras) record three channels per pixel in the visible wavelength range. In contrast, hyperspectral imaging sensors measure up to several hundred channels per pixel. This allows material-specific differences in surfaces to be distinguished. However, the data volume of hyperspectral images increases enormously due to the additional image information (several 100 MB!). Due to limited transfer rates and storage media volumes, there are therefore restrictions on data transfer and archiving.

However, near-real-time applications in particular are becoming increasingly relevant in today's everyday life, which is why the fastest possible data transfer of large volumes of data is also becoming more important. However, as long as only a limited bandwidth (3G, 4G, 5G, ...) is available for this, the alternative is to compress the data. With lossless compression methods, the data is reconstructed identically to the original, but the performance or compression rate is severely limited. The aim is therefore to develop lossy approaches that are as powerful as possible and still cause only a small loss of information.

To this end, Fraunhofer IOSB has developed a 1D-Convolutional Autoencoder architecture that enables compression rates of between 4-100. The encoder compresses the hyperspectral data on the flight platform by combining it with data from different hyperspectral data sources.

Further information:


Scene analysis department

Would you like to find out more about our scene analysis products? 

Keeping an overview with AI

Michel, Andreas & Gross, Wolfgang & Hinz, S. & Middelmann, Wolfgang. (2022). ARM-NMS: SHAPE BASED NON-MAXIMUM SUPPRESSION FOR INSTANCE SEGMENTATION IN LARGE SCALE IMAGERY. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. V-2-2022. 291-298. 10.5194/isprs-annals-V-2-2022-291-2022.

Michel, A., Gross, W., Schenkel, F., & Middelmann, W. (2022). Class-aware object counting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 469-478).

Save data volume with AI

Kuester, Jannick, Wolfgang Gross, and Wolfgang Middelmann. "1D-convolutional autoencoder based hyperspectral data compression." The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 43 (2021): 15-21.

Jannick Kuester, Johannes Anastasiadis, Wolfgang Middelmann, Michael Heizmann, "Investigating the influence of hyperspectral data compression on spectral unmixing," Proc. SPIE 12267, Image and Signal Processing for Remote Sensing XXVIII, 122670H

J. Kuester, W. Gross, S. Schreiner, M. Heizmann and W. Middelmann, "Transferability of Convolutional Autoencoder Model For Lossy Compression to Unknown Hyperspectral Prisma Data," 2022 12th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Rome, Italy, 2022, pp. 1-5, doi: 10.1109/WHISPERS56178.2022.9955109.

Jannick Kuester, Wolfgang Gross, Michael Heizmann, Wolfgang Middelmann, "Impact of different compression rates for hyperspectral data compression based on a convolutional autoencoder," Proc. SPIE 11862, Image and Signal Processing for Remote Sensing XXVII, 118620H (12 September 2021)