About autonomous diving robots, underwater imaging and maritime situational awareness

Our broad range of expertise in underwater sensor technology, sensor data fusion, environmental sensing, autonomous navigation, and remotely operated underwater vehicles enables us to advance the exploration of maritime issues. With our expertise, we move on and under the water to solve different tasks there. Our systems are used for precise mapping of the seafloor, discovery of historical finds underwater, removal of microplastics in the sea, water rescue with diving robots, optimization of underwater images, digitization and automation of waterways or Maritime Situational Awareness (MSA).  

 

These are our service areas:

(Using the links you will jump directly to the corresponding section further down the page.)

  1. Surface and underwater vehicles of various degrees of autonomy (remotely operated vehicles, ROV; autonomous surface/underwater vehicles, ASV/AUV), equipped with cameras or multisensory for different missions ranging from drowning rescue, mapping, object detection and surveying to inspection and maintenance tasks.
  2. Optimization of underwater imagery and exploration of underwater optical communications.
  3.  Maritime situational awareness and detection of their objects using AI-based methods, among others.

We develop autonomous diving drones and surface vehicles

Privacy warning

With the click on the play button an external video from www.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: https://policies.google.com/privacy

Watch our video about the autonomous underwater vehicles of Fraunhofer IOSB-AST (video in German language)

We have many years of experience with remotely operated vehicles (ROV) and autonomous underwater vehicles (AUV) as well as unmanned surface vehicles (USV) as sensor carriers for exploration and inspection tasks. Typical application scenarios include automated mapping above and below water, inspection and maintenance tasks, and underwater archaeology. Our research topics include conception, construction, sensor equipment and control of the (sub)water vehicles, mission planning and navigation as well as sensor data evaluation and fusion.

© Fraunhofer IOSB-AST
An agile, multifunctional and remotely controlled underwater vehicle of the TORTUGA ROV family
© 2020 Google,GeoBasis-DE/BKG, GeoContent, Maxar Technologie
3D overwater map generated in the project TAPS

The different missions of our ROVs and autonomous vehicles:

  • Monitor water quality, take water samples, inspect underwater structures or support fish farming. The agile, remote-controlled underwater vehicles of the TORTUGA ROV family, which unlike conventional ROVs are equipped with an internal energy storage system, are used for this purpose.
  • Exploring the seabed with high-resolution imaging sensors and underwater drones. In the process, Fraunhofer IOSB-AST's DEDAVE vehicle was able to discover historical finds from the Bronze Age and a flight model of the Canadian jet fighter CF-105 Avro Arrow, which was thought to be lost, at the bottom of Lake Ontario. Fraunhofer IOSB's underwater drone technology also held its own in the development of a lightweight technology for mapping the deep sea: our ARGGONAUTS team finished among the world's top five teams in the Shell Ocean Discovery XPRIZE technology competition
  • Saving people from drowning. Real-life experiments by Fraunhofer IOSB-AST to develop a water rescue robot (television broadcast in German) are already underway in swimming pools, and rescue in inland waters is being planned.
  • Precisely measure waterways above and below the surface and thus support the Digitalization of water infrastructure. In a pre-research project, we are developing a autonomous surface vehicle for underwater and overwater mapping – a catamaran equipped with sensors that automates the navigation and mapping of water surfaces or waterways. In the process, potential obstacles are detected and avoided.

On a special maritime mission

We contribute our competencies to these cross-institutional initiatives:

Smart Ocean Technologies

The Fraunhofer research group Smart Ocean Technologies SOT develops future-oriented underwater technologies for sustainable use of the oceans at the Rostock site. The Departments of Systems for Measurement, Control and Diagnosis and Underwater Robotics are involved.

 

 

PLS - Germany's AI platform

With the leading participation of Fraunhofer IOSB, WG7 “Hostile-to-live Environments” of the Learning Systems Platform is investigating how autonomous systems can relieve humans of stressful and dangerous tasks - even under water (application scenario, pdf file).

Inspect, see and communicate under water

Results from Underwater-Vision: Here an image capture in raw state
... and the automatically restored image capture.
© Fraunhofer IOSB
A panoramic image consisting of about 2000 composite frames of a lake bottom.

Inspection: Our autonomous underwater vehicles are helpful for exploration and inspection tasks. For example, various vehicles in terms of equipment, size and payload are used in the maintenance of offshore wind turbines under water. Oil and gas pipelines, submarine cables between continents, and sheet pile walls in ports and inland waterways can also be maintained with our practical autonomous diving systems.

Vision: We use new variable image acquisition and processing techniques to increase image quality and visibility underwater. This enables visual-optical inspection of underwater infrastructure even in turbid waters. In addition, a carpet of images can be assembled from many individual images or videos to form a panoramic image - without unnatural image distortions. These overview maps are used to inspect underwater infrastructure (e.g., bridge foundations), document, archive, and detect changes from older overviews.

Our AI-based classification and detection algorithms for image datasets can help detect creatures or objects of interest on the seafloor(German text). For this, we trained an algorithm that uses artificial neural networks to search for and detect targeted structures on the seafloor from large image datasets.

Communication: Can submarines communicate via laser light, and what impact does water turbulence have on this type of communication? To investigate these questions within the framework of our long-standing expertise in adaptive optics, since 2020 we have a water turbulence laboratory that is unique in Europe (see interview in German on page 3 in InfOSB 1/2020 [pdf]). Special mirror devices in the 500-liter water tank stretch the length of the light beam many times over.

Maritime Situational Awareness

© Fraunhofer IOSB
The XAI toolbox provides the user with a variety of plots, e.g., feature importance and heat plots as explanations for model prediction.

We not only develop underwater vehicles for diverse missions, but also offer assistance systems for decision support. Especially with the advent of high data volumes, AI-based procedures support situation awareness and anomaly detection in the maritime context.

In project MARISA, for example, a maritime surveillance system with automatic procedures was developed to ensure applicable safety requirements and environmental protection. In doing so, human decision makers are supported in focusing attention on relevant information.

The XAI toolbox can be used to quickly evaluate various XAI methods for AI procedures. Among others, surveillance bodies of maritime areas or ports can apply this when it comes to identifying ship types based on trajectories.

In the European defense research project OCEAN2020, 43 partners from 15 EU countries are integrating flying, floating and diving unmanned systems as sensor carriers in fleet units to set up a networked maritime surveillance and reconnaissance mission of the future. Fraunhofer IOSB is contributing its expertise at various levels: From information fusion and tracking, video analysis, development of autonomous water and underwater vehicles and coordination of the Water Strider's swarm behavior to networking and interoperability to situation display and maritime situation analysis.

Our laboratories – built close to the water

In addition to research in open waters, Fraunhofer IOSB has special laboratories for underwater systems at three sites.

Ilmenau

The test basin of our institute branch for applied systems engineering IOSB-AST in Ilmenau with 12m x 8m x 3m offers optimal conditions for the testing of diving robots like DEDAVE, the water rescue robot and the TORTUGA ROV vehicles.

Ettlingen

In the stable 500-liter tank, we explore underwater communication with light. This is sent back and forth many times by means of mirrors, which makes the measuring distance sufficiently long. A heating and a cooling machine provide controllable turbulence.

Rostock

The Smart Ocean Technologies SOT research group in Rostock has direct access to the “Digital Ocean Lab”, an underwater test field in the Baltic Sea. There, smart underwater systems for various purposes can be tested under realistic conditions.