FUTURE HUMAN ACTIVITY PREDICTION USING WAVELET AND LSTM
Keywords:
Deep Learning; predict_future_activity; Convolution Neural Network; Long Short-Term Memory; Wearable Sensor; Recurrent Neural Network, Human Activity Recognition.
Abstract
Future Human Activity Prediction holds significant importance as it enables early detection and monitoring of various aspects, such as elderly care, early fall detection systems, smart-home applications, and E-health monitoring.
A pioneering approach has been developed to achieve this, incorporating the Wavelet transform preprocessing technique for dimensional reduction through signal decomposition. This is followed by the implementation of a deep learning model supported by time series data, enabling real-time monitoring of physical activity.
A novel method has been proposed based on wearable sensor data sources, employing LSTM and time series models, and applied with MHEALTH Dataset. This dataset comprises 12 complex activities and sensor-based devices, ensuring the privacy of patients or participants in real-life scenarios. The results demonstrate that the predicted activity of five steps with an accuracy level for the next day’s activity achieved an accuracy of 98%, surpassing the accuracy and complexity compared with state-of-the-art methods.
Downloads
Download data is not yet available.
References
Van Gestel AJR, Clarenbach CF, Stöwhas AC, Rossi VA, Sievi NA, Camen G, et al. (2012)
Predicting Daily Physical Activity in Patients with Chronic Obstructive Pulmonary Disease. PLoS ONE 7(11): e48081. doi.org/10.1371/journal.pone.0048081.
S. Zhang, et al. "Wearable sensor-based activity recognition and prediction for smart healthcare" Published in: IEEE Access, 2018, doi.org/10.1155/2022/1391906.
Magnanimo, Vito, et al. "A bayesian approach for task recognition and future human activity prediction." The 23rd IEEE international symposium on robot and human interactive communication. IEEE, 2014, DOI: 10.1109/ROMAN.2014.6926339.
Mahmud, Tahmida, et al. "Prediction and description of near-future activities in video." Computer Vision and Image Understanding 210 (2021)و doi.org/10.1016/j.cviu.2021.103230.
J. Liang, L. Jiang, J. C. Niebles, A. G. Hauptmann and L. Fei-Fei, "Peeking Into the Future: Predicting Future Person Activities and Locations in Videos," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 5718-5727, doi: 10.1109/CVPR.2019.00587.
K. Xia, J. Huang and H. Wang, "LSTM-CNN architecture for human activity recognition", IEEE Access, vol. 8, pp. 56855-56866, 2020, DOI: 10.1109/ACCESS.2020.2982225.
F. Li, K. Shirahama, M. Nisar, L. Köping and M. Grzegorzek, "Comparison of feature learning methods for human activity recognition using wearable sensors", Sensors, vol. 18, no. 3, pp. 679, Feb. 2018, doi.org/10.3390/s18020679.
Q. Zou, Y. Wang, Q. Wang, Y. Zhao and Q. Li, "Deep learning-based gait recognition using smartphones in the wild", IEEE Trans. Inf. Forensics Security, vol. 15, pp. 3197-3212, 2020, doi.org/10.48550/arXiv.1811.00338.
M. A. Khatun et al., "Deep CNN-LSTM With Self-Attention Model for Human Activity Recognition Using Wearable Sensor," in IEEE Journal of Translational Engineering in Health and Medicine, vol. 10, pp. 1-16, 2022, Art no. 2700316, doi: 10.1109/JTEHM.2022.3177710..
Chaolei Han a, Lei Zhang a, Yin Tang a, Wenbo Huang a, Fuhong Min a, Jun He b.: Human activity recognition using wearable sensors by heterogeneous convolutional neural networks. Expert Systems with Applications, Volume 198, 15 July 2022, doi.org/10.1016/j.eswa.2022.116764.
C. Catal, S. Tufekci, E. Pirmit, and G. Kocabag, “On the use of ensemble of classifiers for accelerometer-based activity recognition,” Applied Soft Computing, vol. 37, pp. 1018–1022, Dec. 2015, doi: 10.1016/j.asoc.201 5.01.025
Y. Zheng, Q. Liu and E. Chen, "Time series classification using multi-channels deep convolutional neural networks", Proc. Int. Conf. Web-Age Inf. Manage., pp. 298-310, 2014.
R. Freitas, M. Terroso, M. Marques, J. Gabriel, A. Torres Marques and R. Simoes, "Wearable sensor networks supported by mobile devices for fall detection," SENSORS, 2014 IEEE, Valencia, Spain, 2014, pp. 2246-2249, doi: 10.1109/ICSENS.2014.6985488.
Hsieh, Chia-Yeh, et al. "A machine learning approach to fall detection algorithm using wearable sensor." 2016 international conference on advanced materials for science and engineering (ICAMSE). IEEE, 2016, DOI:10.1109/ICAMSE.2016.7840209.
C.-Y. Hsieh, C.-N. Huang, K.-C. Liu, W.-C. Chu, C.-T. Chan A machine learning approach to fall detection algorithm using wearable sensor Proceedings of the International Conference on Advanced Materials for Science andEngineering (ICAMSE), IEEE, Tainan, Taiwan (November 2016), pp. 707-710, doi: 10.1109/ICAMSE.2016.7840209 .
Ajerla, Dharmitha, Sazia Mahfuz, and Farhana Zulkernine. "A real-time patient monitoring framework for fall detection." Wireless Communications and Mobile Computing 2019 (2019): 1-13. doi.org/10.1155/2019/9507938.
Y. Li, G. Chen, Y. Shen, Y. Zhu and Z. Cheng, "Accelerometer-based fall detection sensor system for the elderly," 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems, Hangzhou, China, 2012, pp. 1216-1220, doi: 10.1109/CCIS.2012.6664577.
Reyes-Ortiz, J., Anguita, D., Ghio, A., Oneto, L., Parra, X.: UCI Machine Learning Repository: Human Activity Recognition Using Smartphones DataSet. https://archive.ics.uci.edu/ml/ datasets/human+activity+recognition+using+smartphones(2012). Accessed 10 Nov 2020.
M. S. Ryoo and J. K. Aggarwal, "Stochastic representation and recognition of high-level group activities: Describing structural uncertainties in human activities," 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, USA, 2009, pp. 11-11, doi: 10.1109/CVPRW.2009.5204329.
C. A. Ronao and S.-B. Cho, “Recognizing human activities from smartphone sensors using hierarchical continuous hidden Markov models,” International Journal of Distributed Sensor Networks, vol. 13, no. 1, p. 1550147716683687, Jan. 2017, doi: 10.1177/1550147716683687.
Y. Lin and J. Wu, “A Novel Multichannel Dilated Convolution Neural Network for Human Activity Recognition,” Mathematical Problems in Engineering, vol. 2020, p. e5426532, Jul. 2020, doi: 10.1155/2020/542 6532.
C. A. Ronao and S.-B. Cho, “Human activity recognition with smartphone sensors using deep learning neural networks,” Expert Systems with Applications, vol. 59, pp. 235–244, Oct. 2016, doi: 10.1016/j.es wa.2016.04.032.
Wang, S., Zhu, X.: A hybrid deep neural networks for sensor-based human activity recognition. In: IEEE/2020 12th International Conference on Advanced Computational Intelligence (ICACI), Dali, China, pp. 486–491 (2020), doi: 10.1109/ICACI49185.2020.9177818.
Ragab, M. G., Abdulkadir, S. J., Aziz, N.: Random search one dimensional CNN for human activity recognition. In: IEEE In 2020 International Conference on Computational Intelligence, ICCI, pp. 86–91 (2020), doi: 10.1109/ICCI51257.2020.9247810.
Aliakbarian, F. Saleh, M. Salzmann, B. Fernando, L. Petersson, and L. Andersson. Encouraging lstms to anticipate actions very early. 2017, https://doi.org/10.48550/arXiv.1703.07023.
A. Sadeghian, A. Alahi, and S. Savarese. Tracking the untrackable: Learning to track multiple cues with long-term dependencies. In Proceedings of the IEEE International Conference on Computer Vision, pages 300–311, 2017, doi.org/10.48550/arXiv.1701.01909.
T. Shu, S. Todorovic, and S.-C. Zhu. Cern: confidence-energy recurrent network for group activity recognition. In IEEE Conference on Computer Vision and Pattern Recognition, volume 2, 2017, doi.org/10.48550/arXiv.1704.03
Tehrani, A., Yadollahzadeh-Tabari, M., Zehtab-Salmasi, A., & Enayatifar, R. (2023). Wearable Sensor-Based Human Activity Recognition System Employing Bi-LSTM Algorithm. The Computer Journal, bxad035, Sensors 2023, 23, 1275. https://doi.org/10.3390/s23031275 .
Jaramillo, I.E.; Chola, C.; Jeong, J.-G.; Oh, J.-H.; Jung, H.; Lee, J.-H.; Lee, W.H.; Kim, T.-S. Human Activity Prediction Based on Forecasted IMU Activity Signals by Sequence-to-Sequence Deep Neural Networks. Sensors 2023, 23, 6491. https://doi.org/10.3390/s23146491.
Mohd Noor, M.H., Tan, S.Y. & Ab Wahab, M.N. Deep Temporal Conv-LSTM for Activity Recognition. Neural Process Lett 54, 4027–4049 (2022). https://doi.org/10.1007/s11063-022-10799-5
Predicting Daily Physical Activity in Patients with Chronic Obstructive Pulmonary Disease. PLoS ONE 7(11): e48081. doi.org/10.1371/journal.pone.0048081.
S. Zhang, et al. "Wearable sensor-based activity recognition and prediction for smart healthcare" Published in: IEEE Access, 2018, doi.org/10.1155/2022/1391906.
Magnanimo, Vito, et al. "A bayesian approach for task recognition and future human activity prediction." The 23rd IEEE international symposium on robot and human interactive communication. IEEE, 2014, DOI: 10.1109/ROMAN.2014.6926339.
Mahmud, Tahmida, et al. "Prediction and description of near-future activities in video." Computer Vision and Image Understanding 210 (2021)و doi.org/10.1016/j.cviu.2021.103230.
J. Liang, L. Jiang, J. C. Niebles, A. G. Hauptmann and L. Fei-Fei, "Peeking Into the Future: Predicting Future Person Activities and Locations in Videos," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 5718-5727, doi: 10.1109/CVPR.2019.00587.
K. Xia, J. Huang and H. Wang, "LSTM-CNN architecture for human activity recognition", IEEE Access, vol. 8, pp. 56855-56866, 2020, DOI: 10.1109/ACCESS.2020.2982225.
F. Li, K. Shirahama, M. Nisar, L. Köping and M. Grzegorzek, "Comparison of feature learning methods for human activity recognition using wearable sensors", Sensors, vol. 18, no. 3, pp. 679, Feb. 2018, doi.org/10.3390/s18020679.
Q. Zou, Y. Wang, Q. Wang, Y. Zhao and Q. Li, "Deep learning-based gait recognition using smartphones in the wild", IEEE Trans. Inf. Forensics Security, vol. 15, pp. 3197-3212, 2020, doi.org/10.48550/arXiv.1811.00338.
M. A. Khatun et al., "Deep CNN-LSTM With Self-Attention Model for Human Activity Recognition Using Wearable Sensor," in IEEE Journal of Translational Engineering in Health and Medicine, vol. 10, pp. 1-16, 2022, Art no. 2700316, doi: 10.1109/JTEHM.2022.3177710..
Chaolei Han a, Lei Zhang a, Yin Tang a, Wenbo Huang a, Fuhong Min a, Jun He b.: Human activity recognition using wearable sensors by heterogeneous convolutional neural networks. Expert Systems with Applications, Volume 198, 15 July 2022, doi.org/10.1016/j.eswa.2022.116764.
C. Catal, S. Tufekci, E. Pirmit, and G. Kocabag, “On the use of ensemble of classifiers for accelerometer-based activity recognition,” Applied Soft Computing, vol. 37, pp. 1018–1022, Dec. 2015, doi: 10.1016/j.asoc.201 5.01.025
Y. Zheng, Q. Liu and E. Chen, "Time series classification using multi-channels deep convolutional neural networks", Proc. Int. Conf. Web-Age Inf. Manage., pp. 298-310, 2014.
R. Freitas, M. Terroso, M. Marques, J. Gabriel, A. Torres Marques and R. Simoes, "Wearable sensor networks supported by mobile devices for fall detection," SENSORS, 2014 IEEE, Valencia, Spain, 2014, pp. 2246-2249, doi: 10.1109/ICSENS.2014.6985488.
Hsieh, Chia-Yeh, et al. "A machine learning approach to fall detection algorithm using wearable sensor." 2016 international conference on advanced materials for science and engineering (ICAMSE). IEEE, 2016, DOI:10.1109/ICAMSE.2016.7840209.
C.-Y. Hsieh, C.-N. Huang, K.-C. Liu, W.-C. Chu, C.-T. Chan A machine learning approach to fall detection algorithm using wearable sensor Proceedings of the International Conference on Advanced Materials for Science andEngineering (ICAMSE), IEEE, Tainan, Taiwan (November 2016), pp. 707-710, doi: 10.1109/ICAMSE.2016.7840209 .
Ajerla, Dharmitha, Sazia Mahfuz, and Farhana Zulkernine. "A real-time patient monitoring framework for fall detection." Wireless Communications and Mobile Computing 2019 (2019): 1-13. doi.org/10.1155/2019/9507938.
Y. Li, G. Chen, Y. Shen, Y. Zhu and Z. Cheng, "Accelerometer-based fall detection sensor system for the elderly," 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems, Hangzhou, China, 2012, pp. 1216-1220, doi: 10.1109/CCIS.2012.6664577.
Reyes-Ortiz, J., Anguita, D., Ghio, A., Oneto, L., Parra, X.: UCI Machine Learning Repository: Human Activity Recognition Using Smartphones DataSet. https://archive.ics.uci.edu/ml/ datasets/human+activity+recognition+using+smartphones(2012). Accessed 10 Nov 2020.
M. S. Ryoo and J. K. Aggarwal, "Stochastic representation and recognition of high-level group activities: Describing structural uncertainties in human activities," 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, USA, 2009, pp. 11-11, doi: 10.1109/CVPRW.2009.5204329.
C. A. Ronao and S.-B. Cho, “Recognizing human activities from smartphone sensors using hierarchical continuous hidden Markov models,” International Journal of Distributed Sensor Networks, vol. 13, no. 1, p. 1550147716683687, Jan. 2017, doi: 10.1177/1550147716683687.
Y. Lin and J. Wu, “A Novel Multichannel Dilated Convolution Neural Network for Human Activity Recognition,” Mathematical Problems in Engineering, vol. 2020, p. e5426532, Jul. 2020, doi: 10.1155/2020/542 6532.
C. A. Ronao and S.-B. Cho, “Human activity recognition with smartphone sensors using deep learning neural networks,” Expert Systems with Applications, vol. 59, pp. 235–244, Oct. 2016, doi: 10.1016/j.es wa.2016.04.032.
Wang, S., Zhu, X.: A hybrid deep neural networks for sensor-based human activity recognition. In: IEEE/2020 12th International Conference on Advanced Computational Intelligence (ICACI), Dali, China, pp. 486–491 (2020), doi: 10.1109/ICACI49185.2020.9177818.
Ragab, M. G., Abdulkadir, S. J., Aziz, N.: Random search one dimensional CNN for human activity recognition. In: IEEE In 2020 International Conference on Computational Intelligence, ICCI, pp. 86–91 (2020), doi: 10.1109/ICCI51257.2020.9247810.
Aliakbarian, F. Saleh, M. Salzmann, B. Fernando, L. Petersson, and L. Andersson. Encouraging lstms to anticipate actions very early. 2017, https://doi.org/10.48550/arXiv.1703.07023.
A. Sadeghian, A. Alahi, and S. Savarese. Tracking the untrackable: Learning to track multiple cues with long-term dependencies. In Proceedings of the IEEE International Conference on Computer Vision, pages 300–311, 2017, doi.org/10.48550/arXiv.1701.01909.
T. Shu, S. Todorovic, and S.-C. Zhu. Cern: confidence-energy recurrent network for group activity recognition. In IEEE Conference on Computer Vision and Pattern Recognition, volume 2, 2017, doi.org/10.48550/arXiv.1704.03
Tehrani, A., Yadollahzadeh-Tabari, M., Zehtab-Salmasi, A., & Enayatifar, R. (2023). Wearable Sensor-Based Human Activity Recognition System Employing Bi-LSTM Algorithm. The Computer Journal, bxad035, Sensors 2023, 23, 1275. https://doi.org/10.3390/s23031275 .
Jaramillo, I.E.; Chola, C.; Jeong, J.-G.; Oh, J.-H.; Jung, H.; Lee, J.-H.; Lee, W.H.; Kim, T.-S. Human Activity Prediction Based on Forecasted IMU Activity Signals by Sequence-to-Sequence Deep Neural Networks. Sensors 2023, 23, 6491. https://doi.org/10.3390/s23146491.
Mohd Noor, M.H., Tan, S.Y. & Ab Wahab, M.N. Deep Temporal Conv-LSTM for Activity Recognition. Neural Process Lett 54, 4027–4049 (2022). https://doi.org/10.1007/s11063-022-10799-5
Published
2023-12-24
How to Cite
AL-JUAIFARI, M. K. R., & ATHARI , A. A. (2023). FUTURE HUMAN ACTIVITY PREDICTION USING WAVELET AND LSTM. Journal of Duhok University, 26(2), 541 - 550. https://doi.org/10.26682/csjuod.2023.26.2.49
Section
Pure and Engineering Sciences
It is the policy of the Journal of Duhok University to own the copyright of the technical contributions. It publishes and facilitates the appropriate re-utilize of the published materials by others. Photocopying is permitted with credit and referring to the source for individuals use.
Copyright © 2017. All Rights Reserved.
