Detection of moving objects with a camera on a moving platform (m3motion)

About m3motion

The system m3motion is able to detect and track moving objects with cameras on a moving platform (MTI – Moving Target Indicator). Especially for security and surveillance tasks is it useful to track the motions of persons and objects or to generate mosaic images to allow human observers a better interpretation of the situation.

By estimating  the camera motion continouslywe can stabilize or simply supress the background. We are able to exploit video streams or IR-sensors. The system runs in real-time (25Hz) on a standard PC hardware (>1.7GHz)

Optimum image recognition when the camera and target object(s) move

Application areas for image recognition systems

EO sensors are used in many moving sensor carriers in the fields of "Security" and "Surveillance" to detect dynamically changing scenes:

  • Helicopters perform police duties and monitor borders or capture dynamically changing scenes in general
  • For reconnaissance and surveillance purposes, the Bundeswehr uses unmanned aerial vehicles (drones) with imaging sensors in its worldwide operational areas
  • Imaging sensors can also be found in many all-terrain land vehicles

Optimum image recognition in practice: the problems

Even stationary cameras are not always resistant to vibrations. Fluctuations of masts, self-movements of the sensor (e.g. via pan/tilt device) or of the sensor carrier, but also changes in the optics (zoom) make optimal image recognition more difficult.

In such cases, the human expert cannot always capture the dynamics of the photographed scene with sufficient precision and reliability. Moving target objects are overlooked or already captured objects are lost sight of again. Movements of the sensor itself (e.g. via the pan/tilt mechanism), the sensor carrier or changes in the optics (zoom) make optimal image recognition more difficult.

Support from existing image recognition systems

It is desirable to support the human expert with appropriate functionality. Previous systems are however very limited in their performance. Either they require highly precise information about the flight and recording parameters as well as assumptions to be made about the scene (background) or similar, or they are only able to make rough estimates of relative movements.

Approach and application areas

Robust methods estimate projective images between successive images of an image sequence. Based on this, differential images are calculated, whereby the changes in the images caused by camera movement (translation and/or rotation) or changes in optical properties (zoom) are compensated for. Subsequently, an adaptive thresholding method can be used to detect such scene objects that move relative to the background. The adaptive thresholding method also operates very robustly and is almost independent of contrast.

Even in non-cooperative scenarios, in which e.g. target persons want to remain undetected and try to hide themselves by clothing and movement patterns, the system acts like an MTI method according to the motto "whoever moves is detected".

The human expert can freely adjust the sensitivity of the system. Thus the operator decides whether he is interested in "fast", "moderate" or "slow" movements of potential targets. Apart from this parameter, however, the procedure is parameter-free. For example, the detection of a moving person or a slow moving vehicle is possible without individual parameterisation. To ensure that a target object, once detected, is not lost due to occlusion and/or interruption of movement, a real-time multi-target tracker is coupled. The number of simultaneously detectable target objects depends only on the performance of the computer.

This technology can handle three tasks at once:


Object tracking, Moving Target Indicator

The following film sequence shows an example of the use of the m³motion® system for vehicle and person tracking.

The right-hand area represents the scene recorded, with additional displacement vectors from image to image shown in green and the trajectory of a target object moving relative to the background shown in yellow. The left area shows the current detection of the moving target (red circle).

Automatic image carpet generation

(Image mosaic, panorama images) The system can also be configured so that the object background is not suppressed while the camera is moving, but the image is stabilized.

The images continuously generated from different parts of the scene are combined into a carpet of images which is updated in steps. The larger context thus displayed allows for improved interpretation of the scene, both by the human observer and by automatic procedures.

Image stabilization

By estimating the sensor's own movement, it is also possible to eliminate blurring and stabilise the image in real time and with high precision. In addition, the resolution of a particular snapshot can be subsequently improved by stacking several individual images from a film sequence. One thinks here of recordings from surveillance cameras at ATMs, which operate at very low frame rates.

Product maturity

The m³motion® system will soon be integrated into an image processor of a police helicopter and is about to be introduced to surveillance robots for private security services. Here, m³motion® is typically an automatic component in an interactive system for image-based surveillance.


Department VID of Fraunhofer IOSB

Would you like to learn more about our topics in the area of "video evaluation systems"? Then visit the page of our VID department and get more information.



Further projects of the VID department

Would you like to get to know more projects and products in the field of "video evaluation systems"? Then visit the project page of our IAS department and get more information.