Sensor Data and Information Fusion

Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE

Sensordaten- und Informationsfusion
© Fraunhofer FKIE

Cognitive Tools for intelligent insight

Military reconnaissance systems operating within a networked system are not only mobile but also geographically dispersed. This raises questions such as: how can they be optimally employed and how can information important to the application be extracted from the data streams? Also, many people in the vicinity of potential targets are innocent bystanders on their way to work or going shopping so how can the threat of an attack by any of these people be detected on any given day? These two examples suggest that when carrying out modern security tasks human beings come close to their natural limits of perception as well as their ability to concentrate. The answer is in modern, multi-sensor reconnaissance systems which filter and evaluate the sensor data and information from the databases and fuse it into manageable knowledge. This is the only way that the people involved can gain a comprehensive awareness of the situation and so have a solid and reliable foundation on which to base their decisions. Incidentally, this approach is also becoming important for civilian decision-makers. When an asymmetric threat is posed by terrorist networks we rely heavily on threat detection technologies, particularly when it comes to homeland security. The dual-use nature of the applications developed by the FKIE is self-explanatory. 

Approach

All living organisms rely on sensor data and information fusion. They each have their own way of connecting the messages they receive from their various sense organs with previously learned practical knowledge and with messages from other living organisms. In this way they obtain a picture of their surroundings which is the prerequisite for inducing behavior specific to the situation. It is important within any scientific engineering discipline that the sensor data and information fusion comprehends the linking function so that it becomes automatic; enhancing the knowledge by effective algorithms and incorporating innovative information sources. In application oriented research projects we develop appropriate methods and engineered assistance systems for fusion tasks. Distributed networks of different types of sensors, large databases of background information and, last but not least, interaction with human beings and their practical knowledge are also taken into consideration. The fusion systems resulting from our work are, in a sense, “cognitive tools”, that enhance the awareness of those involved, such as the ability to detect threats in crowds of people. Our modus operandi is characterized by the principle of “Proof of concept”. Methods of resolution are initially sounded out theoretically before being examined in extensive simulation tests and then evaluated and refined in order to construct a final experimental system. Typically the work is targeted towards participation in extensive experiments and proof of capability. In many cases experimental systems and practical experience are the stepping stones to application-relevant solutions in collaboration with industry.

Areas of Expertise

The development of reconnaissance, security and other assistance systems places stringent demands on science. The principal task is always that of reproducing a complex and dynamic operational environment on a computer which has a threat spectrum that is difficult to calculate. It is in this way that computer-supported evaluation is made accessible. At the common interface between sensors, command information systems and decision-makers the key central role is played by sensor data and information fusion. The predominant challenge is to acquire high quality information with intelligent mathematical processes from often inaccurate, incomplete, contradictory or even false data and to then efficiently implement fusion using information technology methods. The staff in the SDF department have a wealth of engineering and scientific know-how as well as expertise in both mathematics and information technology. They are well qualified to master the scientific and engineering challenges they face. They are fully conversant with all of the theoretical and application-relevant aspects of sensor data and information fusion and are able to translate them into practical individual solutions. What is more, their expertise has evolved to such a level over many years that a critical mass has now been reached enabling them to demonstrate our know-how in the form of application-oriented experimental systems or large experiments.

Application

The SDF department’s original area of application research was in support of the Bundeswehr and its Network Enabled Capabilities (NEC). The work is important because sensor networks, and the sensor data and information fusion they are based on, are the informational prerequisites that allow everyone participating in an operation to obtain the timely tactical information that puts them in a position to act in a coordinated manner. Also important is the protection of security-critical access areas (for example to military bases) by means of “Multi-Person-Tracking”, which makes possible the detection of anomalies i.e. threats in crowds of people. These technologies are also now being employed in a civilian context. Terrorist attacks have increased dramatically in recent years, to the extent that certain public infrastructures can become a preferred target of asymmetrical threats. Security assistance systems which continuously and anonymously analyze crowds in places such as airport terminals and railway stations, are now considered to be necessary technical support in the interests of homeland security. They release people from mind-numbing routine checking, so that those deployed can concentrate on the individual surveillance of potential threat carriers. At the same time specific sensors can open up entirely new levels of perception. On account of well-founded threat scenarios the sensory verification of liquid explosive materials or radio-active materials, for example, has become a vital capability of modern security assistance systems. We have now reached a stage where inexpensive sensor and communication technology based on military technology is becoming increasing available for use in the private sector, a good example being for the protection of off-shore wind power installations.

Projects

Fusion 4 Ground Surveillance – Wide-area Surveillance

The ability to keep even the most complex of events under surveillance is essential to the decision-making process at all levels of the chain of command. As a component of the Network Enabled Capabilities (NEC), sensor networks are at the very heart of this capability along with the sensor data and information fusion processes they are based on. [more]

Security Assistance – Support for greater Security

In order to maintain public safety at a place of operation it is essential to protect security-critical areas such as entries into military bases or civil transport infrastructures. To do this effectively the operational forces need to be reinforced by modern, multi-sensor reconnaissance systems. [more]

Mobile Radio Passive Radar for Surveillance

Mobile radio base stations make it possible to employ particularly innovative surveillance technologies. This is because each base station sends out a clearly defined and uninterrupted signal covering its surrounding area. [more]