^{1}

^{*}

^{1}

^{*}

^{1}

^{1}

Low Resolution Thermal Array Sensors are widely used in several applications in indoor environments. In particular, one of these cheap, small and unobtrusive sensors provides a low-resolution thermal image of the environment and, unlike cameras; it is capable to detect human heat emission even in dark rooms. The obtained thermal data can be used to monitor older seniors while they are performing daily activities at home, to detect critical situations such as falls. Most of the studies in activity recognition using Thermal Array Sensors require human detection techniques to recognize humans passing in the sensor field of view. This paper aims to improve the accuracy of the algorithms used so far by considering the temperature environment variation. This method leverages an adaptive background estimation and a noise removal technique based on Kalman Filter. In order to properly validate the system, a novel installation of a single sensor has been implemented in a smart environment: the obtained results show an improvement in human detection accuracy with respect to the state of the art, especially in case of disturbed environments.

Facilitating older seniors independent living has become an important issue of the current research in the field of assistive technologies fostered by worldwide governments. Indeed, as reported in the technical report of the United Nations [

Low Resolution Thermal Array Sensors (LR-TASs) are very suitable in a home environment for substantial reasons. Thanks to their low-resolution, these sen- sors provide useful data without invading the privacy of the dweller as it could happen using cameras or microphones. Furthermore, these devices are small, cheap, easy to be installed in a normal room, and they can work even in absence of light.

LR-TASs are composed of m × n infrared sensing elements, acquiring the temperature of a two-dimensional area. In this paper, we refer to experiments conducted using the Grid-Eye [

Human localization has several applications in Smart Home Environments, e.g., surveillance, health monitoring, and energy management. In particular, LR-TAS has been used to accomplish different tasks.

The goal of the work proposed by Sixsmith et al. [

Erickson et al. [

In Basu et al. [

Mashiyama et al. [

For sake of clarity and completeness let’s report the crucial passages of the work proposed by Mashiyama et al. with the notation used in the rest of this pub- lication. Given an instant of time t , the frame I ( t ) represents the set of T i , j measurements taken by a LR-TAS sensor at time t , each one related to the co- rresponding ( i , j ) pixel. Fixing a time windows τ , the variance v i , j ( t ) of each pixel with t ≥ τ is computed as it follows:

v i , j ( t ) = 1 τ ∑ k = t − ( τ − 1 ) t ( T i , j ( t ) − T i , j ( k ) ¯ ) 2 , where T i , j ( t ) ¯ = 1 τ ∑ k = t − ( τ − 1 ) t T i , j ( t ) . (1)

If the obtained variance v i , j ( t ) exceeds a given threshold V th , a moving person (walking, sitting, or falling) is detected in the current frame. Conversely, if no movement has been detected, the discrimination between a Stopping per- son or No event is done according to the difference ( T diff ) between a person tem- perature T p and the background temperature T b . Given n temp as the number of pixel covered by a standing person, the average of the first n temp pixels of a frame ordered by descending temperature gives T p . Similarly, the average of the remaining pixels gives T b . Finally, only if T diff exceeds a given threshold T th a standing person is revealed. The authors tested their system in a test bed expe- riment, reporting particularly good accuracy results in classifying the mentioned activities, specially considering just the detection phase, excluding the activity classification method.

Most of the work done in activity recognition and human detection using LR- TAS report experimental data obtained by tests performed in a controlled envi- ronment. However, as highlighted by Sixsmith et al. [

The following method aims at retrieving a probability estimation of the presence of at least one person in the LR-TAS field of view. The main steps of the algo- rithm are summarized in the flow presented in

LR-TAS (Low Resultion-Termal Array Sensor) raw temperature data are charac- terized by the presence of noise perturbing the desired measured signal. These type of sensors usually denote low accuracy on a single measurement: Grid-Eye sensor, for example, report the value within Typ.

evolution process on a single region will be taken as an independent dynamic system and, hence, the measurement made by a single sensing element will be fil- tered independently from other measurements.

Consider a dynamic system S represented as follows:

S : ( x ( t ) = F ⋅ x ( t − 1 ) + ξ ( t ) y ( t ) = s ( t ) + η ( t ) = H ⋅ x ( t ) + η ( t ) , (2)

where s ( t ) is the variable to be estimated, y ( t ) is the value obtained mea- suring s ( t ) which is affected by the measurement noise term η ( t ) , x ( t ) is the state variable at time t , ξ models the process noise, F is the system ma- trix and H is the measurement matrix. The proposed noise removal technique is based on Kalman Filter (KF) [

x ˜ ( t ) = F ⋅ x ^ ( t − 1 ) , (3)

while the state estimate is represented as:

x ^ ( t ) = x ˜ ( t ) ) K ( t ) ⋅ [ y ( t ) − H ⋅ x ˜ ( t ) ] , (4)

where K ( t ) is the Kalman Gain [

In order to obtain the expected value of the measured temperature, separating it from the noise component, we applied the Kalman-Filtering technique described in the previous paragraph. Let the state variable x ( t ) be represented by T i , j ( t ) : the average temperature of the objects placed in the field of view of the sensing element in position ( i , j ) at time t . Similarly, y ( t ) is the measure of T i , j ( t ) as acquired by the sensing element in position ( i , j ) at time t . Finally, in order to get the prediction on the state as described in Equation (3), the system matricies have to be set as follows:

F = [ 1 1 0 1 ] and H = [ 1 0 ] . (5)

Thus, from Equation (2):

T ˜ i , j ( t ) = F [ T ^ i , j ( t − 1 ) Δ T ^ i , j ( t − 1 ) ] = [ T ^ i , j ( t − 1 ) + Δ T ^ i , j ( t − 1 ) Δ T ^ i , j ( t − 1 ) ] , (6)

while the variable to be estimated s ( t ) can be derived from Equation (2):

s ( t ) = H x ^ ( t ) = T ^ i , j ( t ) . (7)

The result of the application of KF to measurements collected by a single sen- sing element is shown in

The fundamental assumption to discriminate humans from the background is that the human temperature distribution has to differ from the ambient temperature distribution. In this condition, the human recognition task converges to the ana- lysis of the difference between the current measurements of the sensor cells and the corresponding values of estimated temperature background. Nevertheless, the

temperature background estimation should adapt to the environmental condi- tion changes that can be relatively rapid.

Assuming that the thermistor measurement of the ambient temperature is almost not affected by the presence of humans, this information can be used as a reference to detect changes in the environmental conditions. Thus, the depen- dence between the background temperature T b ( i , j ) ( t ) of the sensing element in position ( i , j ) and the ambient temperature T a ( t ) is a function f ( T a ( t ) ) = T b ( i , j ) ( t ) , it is possible to compute T b ( i , j ) ( t ) from T a ( t ) . The ana- lysis of the sensor cells and thermistor measurements shows a linear dependence (

T b ( i , j ) ( t ) = f ( T a ( t ) ) = [ 1 T a ( t ) ] ⋅ β . (8)

In order to compute β that globally minimized the least square errors, it is necessary to collect a great number of samples. In the final implementation of the proposed method, to make the learning period shorter and let the back- ground estimation algorithm work on-line, an approximation of β is used [

A = [ 1 T a ( t − 1 ) 1 T a ( t − 2 ) ⋮ ⋮ 1 T a ( t − τ ) ] and B = [ T i , j ( t − 1 ) T i , j ( t − 1 ) ⋮ T i , j ( t − τ ) ] . (9)

β as been computed as it follows:

β ^ ( t ) = A ‡ B , (10)

where τ is a time window and ‡ means pseudo inverse operation.

Thus, the estimated background temperature is computed as it follows:

T ^ b ( i , j ) ( t ) = [ 1 T a ( t ) ] ⋅ β ^ , (11)

while the residual squares are given by:

R ( t ) = ( T ^ b ( i , j ) ( t ) − T i , j ( t ) ) 2 . (12)

Finally, since the estimation of β ^ should involve only the background related measurements, Equation (10) is extended using the analysis of the residual squares:

β ^ ( t ) = ( β ^ ( t − 1 ) , if R ( t ) ≥ R th A ‡ B otherwise . (13)

In order to provide as much information as possible in uncertain situations, the proposed method also computes for every cell the probability of human detection in every instant. For this reason, we modeled the probability function q ( T i , j ( t ) ) , describing whether the measurement belongs to the background temperature distribution T b , as a logistic function:

q ( T i , j ( t ) ) = 2 1 + e k ⋅ R ( t ) , (14)

where k is the steepness of the function and the probability p ( T i , j ( t ) ) that that measurement does not belong to the background distribution is (

p ( T i , j ( t ) ) = 1 − q ( T i , j ( t ) ) . (15)

The proposed method aims to improve the accuracy of Human Detection algo- rithms using LR-TAS data in noisy environments. For this reason, the environ- ment has been perturbed during the experiment using air-conditioning system, appliances, and exposing the sensor to sunlight reflection.

detection: the wall [

• Under real condition, furniture and other objects can obstruct the view of the sensor.

• The movement of a human-coming closer and further to the installation point- influences the amount of pixels representing him/her.

• Since the sensor tends to average the value of the temperature in the observed space, human movement also affects the temperature distribution.

For these reasons, the sensor was mounted on the ceiling at the height of 2.7 m with a resultant detection area on the floor of approximately 9 m^{2} (

We have collected several datasets for a total duration of 4 days. During this pe- riod people were asked to perform everyday activities passing and staying under the sensor. The experiment data have been manually annotated to validate the pro- posed algorithm: every pixel is labeled as “1” if it represents human and “0” other- wise.

The proposed method has been tested on the retrieved datasets.

In order to compare the obtained results with the method that, in our know- ledge, reports the best accuracy value in the literature, we implemented also the

human detection algorithm proposed by Mashiyama et al. [

To compare the results of the work of two algorithms, the measurement is said to represent human when a frame contains at least one measurement whose pro- bability p ( m ( t ) ) > p th . The parameters used in both of the algorithms are pro- vided in

The proposed method shows surprising results in terms of precision and recall, proving that it is able to detect humans even in a noisy environment. The me- thod proposed by Mashiyama instead, reports a low recall value since it is missing a lot of detections, which means high false negative value. Moreover, the number of detections (true positive + false positive) is much lower than in the proposed scenario and it is strictly related to the temperature threshold: in this settings Ma- shiyama method obtains a high precision value. The final measure to compare the performance of two methods is given by the ACCURACY measure:

ACCURACY = TP + TN TP + TN + FP + FN , (16)

where TP, TN, FP, FN are: true positive, true negative, false positive and false ne- gative values.

Proposed method | Mashiyama et al. method | ||
---|---|---|---|

10 | 10 | ||

0.01 | 3 | ||

1.5 | 2.5 | ||

0.9 | 0.6 |

Proposed method | Mashiyama et al. method | |
---|---|---|

TP | 0.9988 | 0.3964 |

TN | 0.9408 | 0.9950 |

FP | 0.0592 | 0.0050 |

FN | 0.0012 | 0.6036 |

Precision | 0.9441 | 0.9876 |

Recall | 0.9988 | 0.3964 |

ACCURACY | 0.97 | 0.70 |

We have presented a novel technique to detect humans in indoor environments using Low Resolution Thermal Array Sensor. This approach considers the tem- perature variation in the room due to external dynamics and noise. A Kalman Fil- ter has been used to filter the noise on the temperature measurements while a back- ground estimation technique aims to separate the background from humans.

Final results show an improvement in the human detection accuracy com- pared with the state of the art when performing a field trial in a real environ- ment passing from 70% to 97%.

Currently, the main limitation of the proposed method is that it is hard to dis- tinguish a human presence from other moving heat sources. Further studies in this direction may improve the human detection accuracy in real smart home envi- ronments, reducing the overall system’s false positive rate.

Finally, the mentioned results have been collected using a single sensor insta- llation, however a multisensor system needs to be implemented in order to set up a real scenario in a smart environment. This extension, which requires to handle tech- nical theoretical problems―from placing the sensors to retrieving one overall model with the global state of dwellers and environment―will be part of our future work.

This work was partially financed from project ADALGISA-Regione Lombardia (CUP: E68F13000360009). We thank Dr. Ratti Alessandro from R.S.R. srl (Co- mo, Italy) who provides insight and expertise that greatly assist the research.

Trofimova, A.A., Masciadri, A., Veronese, F. and Salice, F. (2017) Indoor Human Detection Based on Thermal Array Sensor Data and Adaptive Background Estimation. Journal of Compu- ter and Communications, 5, 16-28. https://doi.org/10.4236/jcc.2017.54002