An Activity Recognition System at Home Based on CO2 Sensors

Abstract

Activity recognition of indoor occupants using indirect sensing with less privacy violation is one of the hot research topics. This paper proposes a CO2 sensor-based indoor occupant activity monitoring system. Using the IoT sensor node that contains CO2 sensors, the measured CO2 concentrations in three locations (laboratory, office, and bedroom) were stored in a cloud server for up to 35 days starting July 1, 2023. The CO2 measurements stored at 30-second intervals were statistically processed to produce a heat-mapped display of the hourly average or maximum CO2 concentration. From the heatmap visualizations of CO2 concentration, the proposed system estimated meeting, heating water using a portable stove, and sleep for the occupants’ activity recognition.

Share and Cite:

Matsubara, H. (2023) An Activity Recognition System at Home Based on CO2 Sensors. Journal of Computer and Communications, 11, 64-77. doi: 10.4236/jcc.2023.1111004.

1. Introduction

Japan and many other developed countries in the world are facing a declining birthrate and an aging population. The social challenge is to enhance IoT-based networks to monitor the safety of people living alone, older people, etc. According to the Statistics Bureau of Japan [1] , as of September 15, 2023, there are 36.23 million older adults aged 65 and over in Japan. Among the 200 countries and regions of the world, Japan has the highest elderly population, at 29.1%. Italy ranks second with 24.5%, Finland third with 23.6%, and Germany ninth with 22.7%.

Nishiyama, T. [2] reported that recent developments in human recognition technology have led to the development of indoor activity recognition combined with IoT and machine learning. Activity recognition based on camera images and microphone audio invades the occupants’ privacy. Therefore, recent indoor human activity recognition has proposed indirect methods with low privacy violations. In the following, we describe several studies of indirect activity recognition.

As an example of indirect activity recognition, human activity is estimated from the history of indoor power consumption [3] . Power consumption is measured by magneto resistive (MR) sensors attached to the power lines of home appliances. If MR sensors are attached to home appliances that are used daily, their activities can be recognized. If MR sensors are attached to devices that are used infrequently, the frequency of human activity recognition will be low. The report’s challenge is to increase the perception of occupants’ activity with or without using a particular appliance.

The reference [4] reported a method that uses multiple infrared motion sensors in a building and estimates from their responses. Many infrared motion sensors are mesh-mounted in a building to estimate the spatial distribution of the number of people staying indoors. On the other hand, infrared motion sensors react incorrectly to the presence of small animals, such as dogs and cats, and temperature differences in air conditioner vents. The challenge of the references is to reduce recognition errors and increase the recognition of indoor human activity.

For highly accurate indoor activity recognition by indirect methods, the idea is to turn a living room into a smart house by IoT combining multiple types of sensors. In the report [5] , the sensor node combines three types of sensors: an MR sensor, a door open/close sensor, and an energy-harvesting infrared sensor. The system uses machine learning to estimate the behavior of 10 scenes, including TV watching, eating, bathing, and sleeping. The report’s challenges are to reduce the number of dead and overlapping regions of sensor nodes and to reduce the learning cost of machine learning to estimate activity recognition from sensor node measurements.

The report [6] estimates its life activity scenes from wearable sensor measurements to improve recognition accuracy. They attach a wearable sensor to a person to be watched and estimate the scene of their daily activities. On the other hand, the disadvantage of the wearable sensor is the burden of wearing the sensor 24 hours a day. If the person forgets to attach the sensor or if the sensor’s battery is insufficient, it cannot be recognized. The challenge of the report is to reduce the burden on the person always to wear the wearable sensor.

Matsubara, H. [7] [8] proposed estimating indoor lighting device operation from illuminance sensor measurements. The estimation of indoor subject activity is based on the presence or absence of lighting equipment indoors. To improve the accuracy of activity recognition, it is sufficient if the light sources of only indoor lighting devices operated by the subject to turn on can be measured. If other light sources, such as direct and reflected sunlight and ambient light as noise, are included in the measurement, activity recognition accuracy will be relatively low. For the above reasons, the sensor node should be installed in a location where the light source of indoor lighting equipment can be mainly measured and where sunlight and ambient light from outdoors do not shine directly. The research challenge is to determine the location of sensor nodes to estimate the subject’s operation of lighting devices.

There is an indirect method of estimating indoor occupants by focusing on the indoor CO2 concentration [9] [10] . The indoor CO2 concentration in a closed space is increased by human exhalation. The report [9] describes a method to measure CO2 concentration in a museum with approximately 5000 visitors per day and estimates 5000 visitors based on parameters such as CO2 concentration, indoor ventilation, and room volume. The challenge of estimating indoor stay by CO2 concentration is to apply it to indoor stay in small household use.

We described various indirect activity estimation schemes and related studies [2] - [9] . Survey [11] comprehensively describes studies of activity recognition by the IoT sensor nodes in various IoTs. The reported sensors include image recognition, acceleration, pressure, proximity, magnetism, temperature, etc.

This paper proposes an indoor activity monitoring system using CO2 sensors. The monitoring system aims to recognize the indoor day-by-day activities of a single person or a small group of residents (N = 1 - 8). The IoT sensor node with a built-in CO2 sensor is used to develop an IoT system that stores CO2 concentration measurements on a cloud server. The stored CO2 measurements are statistically processed to estimate indoor residents’ presence or absence and activity status.

The proposed system measured CO2 concentrations in three rooms of 8, 45, and 46 square meters. The CO2 concentrations were measured for up to 35 days starting July 1, 2023, in three rooms: Room A [laboratory], where several students (N = 0 - 8) stay; Room B [office], where one occupant mainly stays during the daytime; and Room C [bedroom] where one occupant mainly stays during the weeknights and holidays. The measurement results are analyzed to estimate the presence or absence of occupants in the room and the ventilation state by opening and closing windows and doors, gas stoves, and other living activities. Finally, we discuss the experimental results and describe the estimation of living activities using the proposed CO2 sensor and the limitations of the estimate.

2. Proposed System

2.1. Outline of the Activity Monitoring System

This paper proposes an activity monitoring system for occupants at home, using a CO2 sensor, which is a low privacy violation. The proposed system has two features.

1) The proposed system estimates the activity of occupants indoors based on changes in CO2 concentration, which increases or decreases with occupants’ exhalation, the opening and closing of doors and windows, and the operation state of mechanical ventilation in the room. The system estimates the date and time a person stays indoors and detects the occupant’s safety daily.

2) The IoT sensor node, including CO2 sensors, can be implemented at a low cost of about $20 U.S. The calibration of the CO2 sensor is set to simply 400 ppm based on outdoor ambient air; the relative value of CO2 concentration after calibration is sufficient for this system to estimate the occupants’ daily activities.

2.2. Activity Monitoring System Using CO2 Sensor

Figure 1 shows an overview of the proposed CO2 sensor-based activity monitoring system. The IoT sensor node (Figure 1(b) and Figure 2) is mounted at a height of 1.5 meters on a wall of the indoor room to be measured at least 1 meter away from the door or window. The IoT sensor node is powered from a commercial outlet in its vicinity. Data to the cloud storage (Figure 1(d)) is transmitted via an indoor Wi-Fi router (Figure 1(c)). This system used Ambient, a cloud storage service offered in Japan. The interval of transmission to the cloud storage was set to 30 seconds. The stored CO2 concentration can be monitored in real-time by a remote observer (Figure 1(e)). In this study, the CO2 concentration stored in the cloud storage is used to determine the presence or absence of indoor residents, to confirm their safety, and to estimate their daily activities.

A block diagram of the IoT sensor node with a built-in CO2 sensor is shown in Figure 2. The CO2 sensor is a Non-Dispersive InfraRed (NDIR) type MH-Z19C

Figure 1. In-home activity recognition system based on CO2 sensors.

Figure 2. Block diagram of IoT Sensor node.

manufactured by Zhengzhou Winsen Electronics Technology Co., Ltd. The DC-DC converter stabilizes the power supply of the IoT sensor node. The embedded microcontroller and development environment are ESP32 with a built-in Wi-Fi module manufactured by Espressif Systems and Arduino IDE 1.8.x, respectively.

2.3. Example of CO2 Concentration Measurement

Table 1 summarizes the three indoor rooms (Room A, B, and C) in which the CO2 concentration is observed, showing the room’s purpose, room size, number of occupants, and main activities of daily living. Figure 3 shows the layout of Room A (laboratory) and the location of the CO2 sensor, and Figure 4 presents a photograph of Room B (office) showing the location of the CO2 sensor, portable stove, and window. Figure 5 shows examples of CO2 measurements in three rooms (Room A, B, and C) over 48 hours.

Figure 5(a) shows an example of a 46-square-meter Room A (laboratory) with seven adults in a meeting from 1 p.m. to 4 p.m. on the first day and eight adults in a meeting from 9 a.m. to 12 p.m. on the second day. The CO2 concentration in Room A increased after the start of the meeting and decreased slowly after the end of the meeting, depending on the exhalations of the residents, the

Table 1. Characteristics of the living rooms for observing CO2 concentration.

Figure 3. Layout of Room A (laboratory) and the location of the CO2 sensor.

Figure 4. Boiling water using a portable stove in Room B (Office).

Figure 5. Examples of recognition of daily activities (meeting, boiling water, sleeping) using the CO2 sensors. (a) Meeting in room A (Laboratory); (b) Boiling water using a portable stove in room B (Office); (c) Sleeping in room C (Bedroom).

opening and closing of doors and windows, and the operation of the mechanical ventilation system.

Figure 5(b) shows an observation example in Room B (office), a 45-square-meter room facing Room A. One resident stayed in this room from 10 a.m. to 6 p.m. on weekdays. The mechanical ventilation is in good operating condition, and there is no significant increase or decrease of CO2 concentration due to the exhalation of the occupant; when the portable heater was used to heat water in Room B, the CO2 concentration increased or decreased in a short period time (about 5 to 15 minutes).

Figure 5(c) shows an example observation of one occupant in an 8 square meter in Room C [bedroom] of his house. During sleep, the bedroom window and door were closed, and the CO2 concentration increased 30 minutes after bed. Upon waking the next morning, the CO2 concentration in the closed bedroom was approximately 1000 ppm. Five minutes after opening the bedroom door, the CO2 concentration decreased to 400 ppm.

Although the measurement results in Figure 5 are only three examples, it is possible to infer whether people are staying or not and their daily activities from the measurement of CO2 concentrations in indoor residences. In this paper, indoor safety confirmation should be able to measure at least one detection of daily living activities associated with changes in CO2 concentration throughout the day. The safety confirmation by the proposed system is assumed to be home if the daily activities can be detected and absent or unknown if they cannot be detected.

3. Experimental Results

3.1. Experimental Conditions

To evaluate the usefulness of the proposed system, the CO2 concentrations in the three rooms shown in Table 1 are measured. The observation period was 35 days, from July 1 to August 4, 2023. The CO2 concentrations in the three rooms were graphed from logs stored every 30 seconds for up to 35 days in the cloud storage. The hourly CO2 concentration visualized by the heat maps was used to estimate the date and time of the resident’s stay, detect safety daily, and estimate daily activities.

3.1.1. Room A (Laboratory)

The observation period was 35 days, from July 1 to August 4, 2023. Room A is on the seventh floor of an eight-story building at the university where the author works, with windows and mechanical ventilation.

Undergraduate students stayed in Room A irregularly during the daytime on weekdays. There were times when no students were in Room A, depending on the day and time. Room A is regularly used for meetings of seven fourth-year undergraduates on Tuesday afternoons and eight third-year undergraduates on Wednesday mornings.

3.1.2. Room B (Office)

The observation period was 35 days, from July 1 to August 4, 2023. Room B faces the corridor to Room A and is occupied by the author alone on weekdays. The residence has a window, a mechanical ventilation system, and a portable stove.

3.1.3. Room C (Bedroom)

The observation period was 31 days from July 5 to August 4, 2023; the missing period was from July 1 to July 4 since the observation started at 10 p.m. on July 4. Room C is the bedroom of the author’s house and is mainly used by the author alone. The residence has a window but is closed during the measurement period. During sleep, the door is closed in addition to the window.

3.2. Experimental Results

Figures 6(a)-(c) present the results of CO2 measurements in Room A, Room B, and Room C, respectively. There were no missing periods during the observation period. The CO2 concentration was approximately 400 ppm when the residents were not present or when the rooms were well-ventilated, and sometimes exceeded 1000 ppm when there were stays in the rooms.

Figure 7 shows the CO2 concentrations averaged hourly and displayed as heat maps; Figure 7(a) is a heat map of Room A. The CO2 concentration was 800 ppm on Tuesday afternoon and Wednesday morning when a meeting was being held, and the CO2 concentration gradually decreased to 400 ppm after the meeting. Figure 7(b) shows the heat map of Room B. Since the residence was well-ventilated and only one resident was staying in the 45-square-meter room,

Figure 6. Measurement results of CO2 concentration in three locations, Room A, Room B, and Room C (July 1, 2023-August 4, 2023). (a) Room A (Laboratory); (b) Room B (Office); (c) Room C (Bedroom) with a missing period from July 1 to July 4.

Figure 7. Heatmap visualizations of CO2 concentration with hourly averages (July 1, 2023-August 4, 2023). (a) Room A (Laboratory); (b) Room B (Office); (c) Room C (Bedroom) with a missing period from July 1 to July 4.

the CO2 concentration was 400 ppm, the same as the outside air, except for some periods. Figure 7(c) shows the heat map of Room C. There were many periods when the concentration exceeded 800 ppm during sleep.

Figure 8 shows a maximum hourly CO2 concentration heat map. Figure 8(b) shows the behavior observed in Room B from 9:00 to 14:00, where a portable stove was used to boil water. Figure 9 and Figure 10 show the mean and maximum CO2 concentrations for each day of the week for a maximum measurement period of 35 days in hourly and heatmap units, respectively.

In Room A, the presence or absence of undergraduate students can be inferred from the heat maps in Figure 9(a) and Figure 10(a). From the heatmap display, it can be assumed that occupants stayed during the weekday daytime and were absent during the weekday nighttime and holidays. The living behavior can be read as having regular Tuesday afternoon and Wednesday morning meetings.

In Room B, the heat maps in Figure 9(b) and Figure 10(b) show that one resident stayed during the daytime on weekdays. In Room C, the heat maps in Figure 9(c) and Figure10(c) show that an occupant was absent during the daytime on weekdays, stayed at nighttime for sleeping, and stayed on weekends. On the other hand, it can be inferred that the occupant do not remain during weekday afternoons.

3.3. Discussion

From the heat map display of CO2 concentrations in the three rooms shown in Figures 7-10, the presence or absence of residents, ventilation conditions in each residence, and living behaviors were inferred.

The CO2 concentration increased with the exhalation of residents and the use of portable gas stoves and decreased rapidly with the release of doors and windows. When there are many residents in a living room, it is easy to infer living behavior because the CO2 concentration tends to increase when the residents are not adequately ventilated. For example, suppose the doors and windows of a residence are closed, and air conditioning is used in summer and winter. In that case, behaviors such as using gas heaters in winter and periodically opening windows for ventilation are easy to infer from the change in CO2 concentration.

In the report [5] , machine learning was used to estimate ten scenes of daily activities, including TV watching, eating, bathing, and sleeping, by combining measurements from three types of sensors. In this study, we believe that if machine learning is applied to the proposed system, it will be possible to automate the estimation of three lifestyle behaviors, such as whether residents stay in their rooms based on CO2 concentration measurements, ventilation such as opening and closing doors and windows, and the use of heaters and portable gas stoves.

The proposed system estimated the presence or absence of each resident, the ventilation status of the residence, and the living behavior from a heatmap display of CO2 concentration using an hourly statistical process as a simple method

Figure 8. Heatmap visualizations of CO2 concentration with hourly maximum values (July 1, 2023-August 4, 2023). (a) Room A (Laboratory); (b) Room B (Office); (c) Room C (Bedroom) with a missing period from July 1 to July 4.

Figure 9. Heatmap visualizations of hourly average CO2 concentration for each day of the week (July 1, 2023-August 4, 2023). (a) Room A (Laboratory); (b) Room B (Office); (c) Room C (Bedroom).

Figure 10. Heatmap visualizations of CO2 concentration at the maximum value per hour for each day of the week (July 1, 2023-August 4, 2023). (a) Room A (Laboratory); (b) Room B (Office); (c) Room C (Bedroom).

that does not require machine learning. The results of the CO2 concentration measurements were used to estimate the living behavior of the residents. In addition, using a portable stove to boil water was shown as an estimation of daily activities for less than one hour. Gas appliances are widely used for cooking in ordinary households. For cooking behavior estimation, the temporary increase in CO2 concentration due to using gas appliances can be visualized as a heat map of the maximum value by statistical processing. The proposed system can estimate more lifestyle behaviors.

4. Conclusions

In this paper, we proposed a system for monitoring activities at home using a CO2 sensor, which is a low privacy violation. The CO2 concentration increases or decreases with a person’s exhalation, the opening and closing of doors and windows, the operating state of mechanical ventilation in a room, and so on. Based on the heatmap display of CO2 concentrations in three residences, the proposed system could estimate daily activities (meetings, sleeping), the date and time of indoor stays, and visualize safety daily.

On the other hand, since the proposed system uses a simple method that does not require machine learning and displays a heat map of CO2 concentration using hourly statistical processing, the types of daily activities that can be detected are limited. Machine learning can be applied to the measurement of CO2 concentration by the proposed system to estimate short-term daily activities, e.g., opening and closing doors and windows, cooking using gas appliances, etc.

Future tasks are to increase the locations where CO2 concentration is observed, such as homes and offices, and to infer more detailed lifestyle behaviors from the long-term observation period. We would like to improve the proposed system based on the findings from the experiments.

Acknowledgements

This study was assisted in part by JSPS KAKENHI Grant Number 21K02914.

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

References

[1] Statistics Bureau of Japan (2023) Statistics Topics No. 138.
https://www.stat.go.jp/data/topics/topi1380.html
[2] Nishiyama, T. (2005) About Special Issue on Developing Technologies for Situational Awareness to Ensure Human Safety. Systems, Control and Information, 49, 147.
https://doi.org/10.11509/isciesci.49.4_147
[3] Tomimoto, H., Miura, K., Haruyama, K., Hitaka, Y., Nakashima, S. and Tanaka, K. (2015) Safety Confirmation of Elderly People by Monitoring Home Electric Appliances Using Magnetic Resistance Sensor. Journal of the Institute of Industrial Applications Engineers, 3, 42-47.
https://doi.org/10.12792/jjiiae.003.02.02
[4] Osaragi, T., Yoshiyuki, N., Kishimoto, M., Oki, T., Iyama, J. and Fukushima, Y. (2022) Estimation of the Number of Walking People in Staircases Using Infrared Human Detection Sensors. Journal of Architecture and Planning (Transactions of AIJ), 87, 1699-1707.
https://doi.org/10.3130/aija.87.1699
[5] Kashimoto, Y., Hata, K., Nakagawa, E., Suwa, M., Fujimoto, M., Arakawa, Y., Shigezumi, T., Komiya, K., Konishi, K. and Yasumoto, K. (2017) A Living Activity Recognition System Based on Power Consumption of Appliances and Energy Harvesting PIR and Door Sensors. IPSJ Journal, 58, 409-418.
[6] Bianchi, V., Bassoli, M., Lombardo, G., Fornacciari, P. and Mordonini, M. (2019) IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment. IEEE Internet of Things Journal, 6, 8553-8562.
https://doi.org/10.1109/JIOT.2019.2920283
[7] Matsubara, H. (2023) An Activity Recognition System at Home Based on Illuminance Sensors. Electronics and Communications in Japan, 143, 754-759.
https://doi.org/10.1002/ecj.12423
[8] Matsubara, H. (2023) An Activity Recognition System at Home Based on Illuminance Sensors. IEEJ Transactions on Electronics, Information and Systems, 143, 754-759.
https://doi.org/10.1541/ieejeiss.143.754
[9] Seong, Y.A., Nariya, T., Hashida, T. and Naemura, T. (2012) Estimating the Number of Visitors in Museum based on CO2 Densitometry. Transactions of the Virtual Reality Society of Japan, 17, 129-132.
https://doi.org/10.18974/tvrsj.17.2_129
[10] Rastogi, K., Lohani, D. and Acharya, D. (2021) Context-Aware Monitoring and Control of Ventilation Rate in Indoor Environments Using Internet of Things. IEEE Internet of Things Journal, 8, 9257-9267.
https://doi.org/10.1109/JIOT.2021.3057919
[11] Babangida, L., Perumal, T., Mustapha, N. and Yaakob, R. (2022) Internet of Things (IoT) Based Activity Recognition Strategies in Smart Homes: A Review. IEEE Sensors Journal, 22, 8327-8336.
https://doi.org/10.1109/JSEN.2022.3161797

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.