Fusion of thermal and visible colour images for robust detection of people in forests

Abstract

Safe operation of automated robotic platforms in environments where humans also work require on-board sensors that can accurately and robustly detect humans in the environment so that appropriate action can be taken. This is a challenging problem in unstructured outdoor environments as most sensors are negatively affected by changing environmental conditions like ambient light and moisture. Our aim is to use a combination of thermal and visible colour images to detect humans in forest environments. The system should be able to work through dense foliage and should not be confused by other objects that generate heat like machines or other animals. We developed and tested a system on a data-set of sensor data collected in a similar outdoor environment but with synthetic targets added to highlight the ability of the system to be robust to severe optical occlusion in dense vegetation and to the presence of hot machines that could fool the thermal sensor. Our initial results show promise and also highlight where improvements can be made with further testing in more realistic forest environments.