Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB



Automation meets microscopy

Microscopic inspection systems are getting more and more important in assuring quality of industrially produced goods. With its newly built up microscopy laboratory (MicroLab) Fraunhofer IOSB is taking a new approach. Various microscopes are combined with standard automation components, such as robots, positioning stages and illumination components, to realize automatic industrial inspection (A1). MicroLab is equipped with versatile optical microscopes which provide a wide range of sensors for combined usage. Currently MicroLab contains a macroscopic device with different illumination components, a research microscope with UV illumination, a white-light interferometer and a 3D reconstructing autofocussing system, which can also be used for roughness measurements.



The key role in automation in MicroLab is played by a six-axis industrial robot. The robot, which is connected to a central control unit that interacts with additional sensors, places various specimens onto the microscopic devices in an exactly defined position. The inspection system therefore exactly knows about the overall condition of a currently running inspection process. It can intelligently influence this running process to optimize throughput, for example using all inspection devices in parallel. It can also prioritize specific specimen, giving it a decisive advantage over industrial assembly lines, which can handle goods only sequentially.



Analysis of wear and friction on a mechanical specimen


General applications

MicroLab deliberately uses standard industrial automation components. These can be combined at will to realize customer-driven inspection systems for visual inspection and quality assurance analysis. The different sensor components with their diverse properties can be used to examine the properties of a large number of different specimens, such as completeness of electronic components or roughness of iron, steel or other industrially produced materials. The combination of multiple microscopes brings inspection tasks to a new level, allowing specimens to be inspected by multiple sensors with a range of characteristics within one automation process. The gathered information can then be used as input for multi-sensor fusion. Figure B1 shows the effect of wear and friction within an industrial process on a mechanical specimen. The microscopic devices are  further used to acquire ground truth information. This is extremely important when constructing new sensor components in order to back up the measurement results. Another reason our customers like the MicroLab concept is its modular construction. It is quite easy to integrate components of MicroLab into existing quality assurance systems.


Transfer of established image processing algorithms into microscopy

The long-term knowledge of IOSB in the fi eld of image processing is of high value in the context of MicroLab. Established methods of image processing, ranging from image fusion to photometric stereo, are transferred to the microscopic dimensions. The integration of such established methods is the basis of providing modular microscopic inspection systems for industrial purposes. Especially image fusion is a highly appreciated approach to increase the depth of field. As the depth of fi eld becomes quite small at higher levels of magnifi cation it is important to compute a synthetically enhanced image that has all focal planes of interest in focus at once. To achieve this, the software creates a depth map in the background which, together with the recorded image series, provides the input for the image fusion algorithm. Figure B2 gives an overview of the overall fusion process. MicroLab, with its multiple sensors, also delivers great possibilities for research and education. Over the past year a novel approach for instrumenting a microscope with gaze-based interaction was developed.


Industrial microscopy as a service

Although conceived only recently, MicroLab has already established itself as a contact point for a wide range of applications (B3). MicroLab offers services for customers to inspect surfaces with different microscopes in order to gain novel information of their specimens. This information can be used to develop new methods and inspection systems for Quality Assurance in an industrial domain.


MicroLab in action           © Fraunhofer IOSB, photographer: Indigo Werbefotografie




MicroLab live:



Microscopy laboratory MicroLab - live at CeBIT 2014 in Hanover:




A product of the business department Mess-, Regelungs- und Diagnosesysteme (MRD)