Improved visibility when driving complex vehicles

The Bundeswehr operates more complex and heavy land vehicles with limited visibility to the exterior than any other organization. The safety measures common in these vehicles, such as heavy armor and small, concealed windows paired with the fact that they are frequently operated »under hatch«, put high mental and physiological demands on their drivers.

The vehicles operate between the conflicting priorities of combat strength, maneuverability, crew protection and safety for both friendly forces and the civilians. This applies not only to vehicles that are specifically designed for combat operations, but also to specialist and transport vehicles that are not only exposed to the dangers of enemy fire and improvised explosive devices (IEDs) but also to complex traffic situations.

This situation is further complicated by the fact that today's military vehicles have to be designed for both symmetrical and asymmetrical warfare. As a consequence of the crew protection, the visibility to the exterior is significantly reduced, which subsequently also reduces the orientation and situational awareness of the crew,  making it impossible to be fully aware of their surroundings.

Electronically enhanced vision offers a potential solution to this problem. Market-ready assistance and vision systems already exist in civilian road vehicles. Within the framework of the »Vision support and driving assistance for protected vehicles (SiFaU)« project, Fraunhofer FKIE is examining the extent to which these systems can also be used in the context of military vehicle guidance. In addition, it is aiming for a more extensive multimodal support of the vehicles.

The aim of the »SiFaU« project series is to research technical assistance systems which support the driving task in complex situations in military vehicles. The central research task and challenge is that the visual assistance systems and the non-visual driving assistance systems to complement each other ergonomically.

Within the »SiFaU1« project the design and use space was structured, an initial market survey and market research was carried out, basic concepts were sketched out and in an initial simulator test two variants of machine vision were compared with a simulated baseline. An early version of the machine vision system was also implemented for a trial in a comparatively simple vehicle, the armored reconnaissance vehicle »Fennek«. The results obtained were so encouraging that further research was initiated, aimed at further investigating the feasibility and usability as well as the transferability to other vehicles.

In the follow-up project »SiFaU2«, further research was conducted on the previously positively evaluated vision assistance system, which was then also evaluated in a number of simulator and real-vehicle tests. This project also explored the initial integration of a rear-view mirror camera. Additionally the machine vision concept extended by an augmentation of the view with projections of roadways and hazard markings, among other things, was also evaluated in the simulator. Experienced Bundeswehr drivers then tested this enhanced vision concept in a real vehicle.

To support collision avoidance, an initial combination of machine enhanced vision and a driving assistance system was developed and applied in the simulator. In parallel, an emergency procedure was drafted in cooperation with the vehicle registration office and an investigation was conducted into the driver's ability to perform an emergency take over in the event of the machine enhanced vision system failing.

Electronic vision is a very current and high priority issue for the German Armed Forces. Consumer-grade driver assistance and vision enhancement systems for civilian vehicles are already being evaluated for their suitability for military applications. Within the framework of the »SiFaU« project series, experienced Bundeswehr drivers have tested a five-monitor system with integrated rear-view mirrors. These trials confirmed that the system offers a better 360° visibility as well as additional key benefits, such as the elimination of large blind spots and the reduction of having to make significant head movements. They further confirmed that the augmentation, which provides a good orientation aid, especially when cornering.

Due to the strong need for a machine enhanced vision, especially when driving under hatch, the Fraunhofer FKIE received the grant for the follow-up study »SiFaU3« as of September 1, 2017. Within the scope of this project, further systems to substitute the direct vision to the exterior, such as a Head-Mounted Display (HMD), were investigated and the existing five-monitor system was extended to include additional views, such as a ramp mirror. The augmented vision concept was further developed, towards also providing a practical support for non-driving related tasks, such as detecting and highlighting enemy units.