Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE
Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE
15604878 - eye viewing digital information represented by ones and zeros

© iStock/Jan Otto

Port surveillance on, above and below the water

Terrorist attacks, drug smuggling, white-collar crime – these are all conceivable scenarios for illegal activities that can occur at sea, on land and in the air in port areas. Unfortunately, this is not only the stuff of fiction but constitutes a real threat and endangers the security situation. As part of a research project funded by the German Federal Ministry for Economic Affairs and Energy (BMWi), the Fraunhofer FKIE is developing a civil port surveillance system in partnership with ATLAS Elektronik and Bremenports.

Situation picture optimization for port protection (LoMA)

Expand all Close all

© Fraunhofer FKIE
A port, such as here in Bremerhaven, is a complex and confusing critical infrastructure. A comprehensive situation picture is needed to assess and ensure safety on-site. So far, such a system has not been available to the supervisory authorities. The BMWi-funded research project LoMA aims to change this.

A port is a complex critical infrastructure. A multitude of people and technologies are involved in its operation and organization. Ensuring port security is an essential aspect of this process. Monitoring a port requires the collection of a great amount of information from above and below the water.

In order to generate a reliable situational picture, this data, which can be collected by different sensors and information sources, has to be merged and further processed. This data fusion and evaluation is the task being addressed by the project LoMA, a German acronym for »Situation Image Optimization for Maritime Awareness«. It is the first project approved by the German Federal Ministry for Economic Affairs and Energy (BMWi) as part of its »Real-Time Technologies for Maritime Security« funding program.

Base scenario: attack from the water

The aim of the research project is to develop a comprehensive port monitoring system. It is intended to help identify and track a suspicious object as early as possible and to analyse its potential threat. Based on proposals, this system should enable security people to react adequately.

The base scenario in this project is a targeted attack on a port by a rubber boat. At present, this would not be detected and could approach and attack unnoticed. In the face of an increased terrorist threat, this type of incident is one of the hazards that port operators and authorities have to contend with, and one that they therefore want to be prepared for.

© Fraunhofer FKIE
For a comprehensive situation picture data from a wide range of sensors – radar, camera, and sonar sensors – must be merged. The picture shows a camera system from Fraunhofer FKIE during a measurement campaign in Bremerhaven.
© Fraunhofer FKIE
Surveillance footage of the camera system.

LoMA is designed to support the early detection of abnormal movements on and under the water, identify the target, and then alert authorities. For this purpose, the system combines data and information from multiple sources into one information layer. In addition to the Automatic Identification System (AIS), a transponder system for the exchange of navigation and other ship data, the system merges the data from radar, camera and sonar sensors as well as ship reporting data. Another focus of the project is on integrated display, alerts, and decision support.

The concept for the monitoring system is divided into three task fields/components:

  • integrated situation picture
  • anomaly detection with early-warning system
  • adaptive decision support

The methods developed were continiously evaluated in joint measuring campaigns conducted by Fraunhofer FKIE and ATLAS Elektronik in Bremerhaven and, hence, in a real port environment. This was made possible by the support of Bremenports through the provision of the infrastructure in Bremerhaven as well as their help with the planning and conducting of the measurement tests.

© Jan-Otto, sorapop/iStock, 123RF Montage
A large, interactive map is a central component of the clearly arranged human-machine interface, a development of the Fraunhofer FKIE. It serves the maximum user-friendly processing of the freshly merged situation picture.
© Fraunhofer FKIE

The main focus of the Fraunhofer FKIE's work in the joint project is information management in the form of sensor data integration, anomaly detection, target evaluation and classification, as well as the ergonomic presentation of all the merged information.

In the interest of creating the most user-friendly display of the merged situation picture as possible, the FKIE team developed a human-machine interface guided by the principles of clarity, simplicity and situational awareness. It is based on practical user requirements, which were determined by surveys, usability studies, experiments, and expert reviews.

»The result is an adaptive, intuitive, and uncluttered interface whose central component is a large map,« explains FKIE project manager Anastasia Schwarze. »Users can interact with it in a variety of ways – they can display or hide information as required, for instance. In addition, warning messages and decision aids are automatically displayed. These are intended to assist in making the safest, most efficient decisions possible. The aim is to provide at a glance all of the relevant information evaluated by the system.«

The next milestone of the joint project is a large-scale test campaign in Bremerhaven under authentic conditions and with real-world scenarios. This is where the system should prove its efficiency.