Occupancy Estimation from Thermal Images

The emergence of intelligent technologies enables smart services in the home environment to provide the residents with convenience and efficiency in our daily life [5]. Many research are based on a single occupant environment [3]. In reality, multiple occupants live in a dwelling. Therefore, a new functional model is needed to determine the number of people in a space, referred to as occupancy estimation. In application, it can help with heating, ventilation, and air conditioning systems control and even security monitoring in the smart buildings [5].

This demo focuses on occupancy estimation using thermal images. Thermal imaging cameras are chosen because they offer advantages over intrusive sensors (such as RGB cameras) in terms of privacy protection and better than non-intrusive sensors (such as passive infrared (PIR) sensors) in terms of estimation results [1]. There are many challenges in this task. One challenge is that many objects, such as the CPU of a computer, are similar in temperature to a human [2]. Another challenge is that indoor environments may have excessive illustrations and far too much thermal noise [5]. We propose a computationally efficient, non-intrusive, and privacy-preserving occupancy estimation system for the smart environment. Our proposed model predicts the number of occupants with an average accuracy of 71.6% in six experiments by combining intensity-based and motion-based human segmentation.

The architecture of the system is shown in Fig. 1. The system has three major components: Data Acquisition, Occupancy Estimation Model, and Labeled Video Generation. Thermal videos from an overhead view can be used as an input to our system. To guarantee the quality of the estimation results, the resolution of the video should be at least 200*100. Furthermore, people should not wear too thick clothes under the camera. In the second component, human segmentation and classification are performed in the occupancy estimation model. Filters are used to determine the classes for the segments, and the number of selected classes determines the occupancy number. As output of our system, a labeled video is generated with predicted values indicating the number of people in the scene. From the video, each frame can be visually inspected by the user.

The proposed estimation model consists of two phases: Preliminary Parameter Configuration and Occupancy Estimation. A process flow that describes the process from the parameter configuration through the occupancy estimation is shown in Fig. 2. We discuss each phase in the following subsections:

We use the Flir FB-Series thermal camera in our experimental setup, which has a relatively high resolution. The experiments have been conducted on a Mac operating system with a Core i3 processor and an 8GB RAM. Six experiments under different scenarios are done to test the effectiveness of our system, shown in Table 1. The output is a video of all frames of size 200*400, marked with the corresponding predicted occupancy, actual occupancy, and confidence scores calculated by $confidence_{i}=1-\left(\frac{estimation_{i}-real_{i}}{real_{i}}\right)$. We estimate the occupancy every two seconds, and the predicted occupancy number is shown in the display’s top left corner. A video demonstrating the system can be found in the following link: https://youtu.be/Av9BkB_ZZJc.

This research was partly made possible by DP160103595 and LE180100158 grants from the Australian Research Council. The statements made herein are solely the responsibility of the authors.