Drone operations in disaster areas

Unmanned Aerial Vehicles (UAVs) are increasingly being used for search-and-rescue purposes because they are capable of quickly flying over large areas of destroyed infrastructure. According to the International Emergency Drone Organization (IEDO), they have already successfully contributed to the rescue of more than 400 people to date. This demonstrates how effective drone technology is at helping to locate disaster survivors and how it significantly speeds up the response time of rescue teams.
Currently, drones that provide support during rescue or firefighting missions typically use imaging sensors (daylight or thermal cameras). But in many disaster scenarios, potential victims are not visible to these types of sensors. For example, victims may be trapped under rubble or may not be visible due to darkness, smoke or fog.
For such cases, Fraunhofer FKIE is currently designing LUCY, the Listening system Using a Crow's nest arraY. Experimental configurations of the system have already been built and are undergoing testing. The Array Signal and Multichannel Processing research group is carrying out the technology evaluation, the setup of the experimental system and the testing.

LUCY is a system that can be attached to drones to determine where certain sounds, including human calls for help, are coming from. The acoustic sensor consists of an irregular volumetric array of microelectromechanical systems (MEMS) microphones known as a "crow's nest array". MEMS microphones are small, lightweight, inexpensive and, most importantly, powerful microphones used in many devices such as cell phones. These types of microphones have been shown to have sufficient sensitivity to detect impulse-like sounds such as screams. They make it possible to build a small, lightweight, and easy-to-carry system, regardless of the type of drone it is attached to.

The human ear is a sensory organ that picks up vibrations and transmits them to the brain. The brain processes the nerve signals and analyzes the received sound information – things like what direction a sound is coming from and what is background noise. LUCY works in a similar way, using microphones to capture sounds and a signal processing unit to analyze the sound information and estimate the direction of incidence of the sounds generated by casualties.

In the work carried out so far, direction-finding and pinpointing impulse noises have already been successfully tested on a Fraunhofer FKIE experimental system with 32 microphones. Due to the miniaturization of the sensor elements, more MEMS microphones can now be used. Currently, the institute is building a 64-microphone demonstrator, which due to its modular design can use all or just a part of the microphone array depending on the application. Methods developed by Fraunhofer FKIE are being used to optimize both detection range and the accuracy of direction-finding and location capabilities in the irregular geometry design for the microphone array.

An adaptive filter is also currently being designed to distinguish between different types of noise. The aim is to suppress noise from helicopter rotors, for example, or wind and rain in order to better detect impulse sounds such as screams and tapping. Initial versions of this filter have already been tested and have performed very well in detecting impulse-like sounds in the presence of other sounds. At the same time, work is being done to implement improved noise detection using artificial intelligence (AI) signal processing to achieve greater accuracy and reliability. This should make it easier to distinguish between screams and other impulse sounds during detection. Using AI-based methods, however, requires training with reference data.

Finally, the angle of incidence of a sound expressed in azimuth and elevation is determined using a technique called "coherent beamforming". By moving the drone through the air in the right way, a person's calls for help can be detected from different directions, making it possible to pinpoint the exact location of the person by intersecting the lines of bearing from the measured angles of incidence. This function is currently being tested.

LUCY is a stand-alone system that can be used as a single sensor. Adding more sensors to a drone equipped with LUCY would be useful to give responders a more complete picture of the situation in the disaster area. For instance, using LUCY in combination with optical sensors would offer advantages. Depending on the situation, thermal imaging cameras could help to find people. In addition to the pure visualization of the position of an injured person determined by LUCY (i.e., camera guidance using LUCY), which is helpful for emergency services, sensor data fusion can often be used to create a more reliable and accurate picture of the situation. In addition, more than one drone equipped with these sensors could also be used to improve the performance of a reconnaissance mission. The Sensor Data and Information Fusion (SDF) department develops and researches special algorithms for such purposes as sensor data fusion, resource management and path planning.

What is unique about this research approach is the use of the well known MEMS microphone technology and its application in a demonstrator to attain new capabilities – in this case, for direction-finding and pinpointing of impulse-like sounds such as shouting. The use of this technology on drones, in particular, has the potential to significantly improve search and rescue operations in large and inaccessible areas or areas with widespread destruction. It is also conceivable that it could be used on other platforms as a complement to existing systems, such as unmanned ground vehicles (UGVs), for the same rescue applications.