Project updates

April 2022

Bulk material data sets are now available online

Large parts of the data sets recorded with an area scan camera as part of the project are now publicly available online. On Zenodo.org, image data of various materials (balls, cylinders, cuboids, peppercorns, wheat, partly recorded on both belt and chute sorters)  as well as measurement data already associated to complete tracks (in .csv format) are now available for download.

---------------------------------------------------------------------------------------------------

September 2021

Particle-specific prediction models with Mixture of experts of Kalman filters and neural networks

Coping with dynamically changing particle streams, such as different materials, compositions, and mass flows, constitutes a major challenge in sensor-based sorting. To allow for robust and highly accurate sorting under these conditions while reducing the operator's workload, we proposed a new method based on the combination of multiple algorithms for determining nozzle timing and nozzle selection in the article "Mixture of Experts of Neural Networks and Kalman Filters for Optical Belt Sorting" (DOI 10.1109/TII.2021.3114282) published in the journal IEEE Transactions on Industrial Informatics. Mixture of experts is a multiple model method from machine learning, which automatically selects one or more models suitable for the current situation, the so-called experts, from a bank of predefined experts and combines their predictions. Our approach therefore combines highly accurate neural networks learned for specific materials and robust approaches such as physical motion models. The results show both improved accuracies in already known situations and improved robustness to new particle types. The approach is thus able to identify the most suitable experts individually for each particle and successfully combines the advantage of data-driven and model-based methods. In particular, the approach reduces the operator's workload, since once the experts have been defined, they are selected quickly and automatically without further human intervention.

---------------------------------------------------------------------------------------------------

September 2020

IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems - MFI 2020

---------------------------------------------------------------------------------------------------

14.-16. September 2020

„Machine learning based multi-object tracking for bulk material sorting"

In order to develop an approach that can autonomously adapt to the sometimes very different motion behavior of different materials, we presented an adaptive algorithm for multi-object tracking in a contribution to IEEE-MFI 2020. Based on recurrent neural networks (RNNs), the motion behavior of the particles to be sorted can be learned on the basis of a training data set. Manual modeling of the motion behavior of the particles is thus no longer necessary. As a special feature in the field of RNNs, the approach also allows the prediction of uncertainties for the tracked particle positions. It was shown that multi-object tracking can be improved compared to the use of RNNs without uncertainty estimation and is on the same level as model-based methods.

more info about the article

------------------------------------------------------------------------------------------------

Berlin / April 2019

Innovationsday Mittelstand des BMWi

Review: The "Future Festival" in Berlin once again demonstrated the wealth of ideas of small and medium-sized enterprises and illustrated the effectiveness of the technology-open innovation support provided by the Bundesministeriums für Wirtschaft und Energie (BMWi).

---------------------------------------------------------------------------------------------------

Berlin / May 9th 2019

GVT - Project of the year 2019

„Improvement of optical bulk material sorting by simulation-based development of tracking methods"

IGF 18798 N

We will present this project at the Innovation Day. Three project partners are involved: Ruhr-Universität Bochum, Institut für Energietechnik, Lehrstuhl für Energieanlagen und Energieprozesstechnik, das Karlsruher Institut für Technologie (KIT), Institut für Anthropomatik und Robotik, Lehrstuhl für Intelligente Sensor-Aktor-Systeme und das Fraunhofer Institut für Optronik, Systemtechnik und Bildauswertung (IOSB).

---------------------------------------------------------------------------------------------------

May 2018

Final report for the project "Inside Schüttgut"

The Inside Schüttgut research project was successfully completed at the end of February 2018. In addition to the description of individual highlights here on the homepage, a final report has now been compiled in which a great deal of information about the work carried out within the framework of the project and the results achieved can be found. The current version of the final report can be downloaded here.

---------------------------------------------------------------------------------------------------

March 2018

Talk at the Sensor-Based Sorting & Control 2018

Our project was on the SBSC 2018 in Aachen with a presentation. Various highlights of the project were presented and clearly summarized in an article.

The article can be can be viewed here.

---------------------------------------------------------------------------------------------------

November 2017

Real-time multi-object tracking for optical bulk sorting

Efficient tracking solutions are needed to simultaneously track the trajectory of thousands of objects. In the article Real-time multitarget tracking for sensor-based sorting: A new implementation of the auction algorithm for graphics processing units (DOI 10.1007/s11554-017-0735-y) in the Journal of Real-Time Image Processing we present a novel approach to quickly solve the association problem, the potential bottleneck in terms of runtime. The clever use of a suitable data structure allows us to solve the problem using the Auction Algorithm in a particularly efficient way on a GPU. The article has been published Open Access and is therefore available to everyone.

---------------------------------------------------------------------------------------------------

November 2017

Use of orientation estimates in the multiobject tracking

With the current state of the art, individual bulk particles are generally not directly distinguishable from each other visually. However, the tracking we are using in this project allows us to deduce from the measurements which particle they originate from by means of models, assumptions and algorithms. However, there is an error rate in this process. This error rate can be reduced by adding another feature that can already be extracted using current image processing algorithms. At the International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) 2017 at Daegu, South Korea, we presented a paper showing that by adding the orientation of the particles the error rate can be reduced. Reducing the error rate in real systems results in an increase of the sorting quality.

---------------------------------------------------------------------------------------------------

September 2017

Talk at the PARTICLES 2017

The numerical approximation of the deflection process has been done by a DEM-CFD coupling. Thus, a complete numerical model of the sorter now exists for the first time. First results and important comparisons between simulations and experiments were presented at the PARTICLES 2017 in Hannover. The conference proceedings are available online at the Website.

---------------------------------------------------------------------------------------------------

March 2017

Talk at the OCM 2017

Within the Inside Schüttgut project, multi-object tracking methods for the optical sorting of bulk materials are implemented, among other things. In addition to increasing the precision of material separation, information about the movement behavior of individual objects contained in the material flow can be derived by tracking their trajectories. On the International Conference on Optical Characterization of Materials (OCM-2017) we presented the first results on this topic (Improving material characterization in sensor-based sorting by utilizing motion information). Particularly pleasing: our contribution was awarded with the Best Paper Award

Update October 2017: A expanded version of the article has now appeared online in the magazine tm technisches messen!

---------------------------------------------------------------------------------------------------

February 2017

Inside Schüttgut at the MFI in Baden-Baden

Our project was represented with two papers and an exhibition booth at the International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) 2016 in Baden-Baden.

When using multitarget tracking systems with hard association decisions for each time step, solving the resulting association problem under fixed real-time conditions is a major challenge. In the framework conference, we presented a system that, depending on the estimated severity of the problem, dynamically selects a solution strategy from a pool of available strategies (Fast Multitarget Tracking via Strategy Switching for Sensor-Based Sorting, DOI 10.1109/MFI.2016.7849538). Thus, in the case of high load situations, potentially lower quality results, which can however be computed faster, can be resorted to.

 A key question of the research project is to what extent the error of physical separation in optical bulk sorters can be reduced by using multitarget tracking methods. We also presented the first investigations on the basis of simulation data at the conference.  (Simulation-Based Evaluation of Predictive Tracking for Sorting Bulk Materials, DOI 10.1109/MFI.2016.7849539).

December 2016

Numerical simulation of optical bulk material sorting systems: DEM & CFD

A deep insight into Discrete Element Method (DEM) simulation as well as the coupling with Computational Fluid Dynamics (CFD) is provided in our new article Numerical modeling of an automated optical belt sorter using the Discrete Element Method, DOI 10.1016/j.powtec.2016.07.018 in the Elsevier Journal Powder Technology. In addition, we present first experiments to reconcile experiments on optical sorting systems and the simulation Numerical Investigation of Optical Sorting Using the Discrete Element Method, DOI 10.1007/978-981-10-1926-5_115. With corresponding results, we were also present at the 7th International Conference on Discrete Element Methods which was held at Dalian University of Technology, China.

---------------------------------------------------------------------------------------------------

February 2016

Predictive tracking based on simulation and experimental data

Predictive multi-object tracking for optical bulk sorting: Within the scope of the project, procedures are being evaluated both on the basis of experimentally obtained data and with the aid of data as obtained from a DEM simulation. This concept as well as first results have now been published in the journal tm – Technisches Messen Improving Optical Sorting of Bulk Materials using Sophisticated Motion Models, DOI 10.1515/teme-2015-0108.

---------------------------------------------------------------------------------------------------

October 2015

Predictive tracking for optical bulk sorting

At the start of the InsideBulk project, we introduced the concept of predictive tracking in optical bulk sorting as part of the International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) 2015 at San Diago (TrackSort: Predictive Tracking for Sorting Uncooperative Bulk Materials, DOI 10.1109/MFI.2015.7295737)