Object Detection in Video Summarization for Video Surveillance Applications
Abstract
For effective Data Extraction (DE) and Data Analysis (DA), the constant flow of visual information offers unique techniques in the Video Surveillance (VS) domain. This VS application demands the significance of advanced Object Detection (OD) for obtaining Video summarization in this study. Then, the accurate detection and the location of objects in the video frames are known as OD, as it is crucial for DE in the VS. To improve OD, the research offers advanced techniques like Faster R- (Convolutional Neural Networks) CNN or FRCNN using Inception ResNet V2 (IR-V2) via the application of CNN, Region Proposal Networks (RPN) and Deep Learning (DL). The empirical outcomes indicate that this suggested framework delivers improved OD accuracy of 93.5% compared to other techniques. By overcoming Big Data (BD) in the modern VS, the combination of sophisticated Computer Vision (CV) techniques with inception modules and residual connections.
References
A. Yali. P. Felzenszwalb and P. Girshick, Object detection. In Computer Vision: A Reference Guide, pp. 875-883, 2021.
A. Karbalaie, F. Abtahi and M. Sjöström, Event detection in surveillance videos: a review. Multimedia tools and applications, vol. 81, no. 24, pp. 35463-35501, 2022. doi: 10.1007/s11042-021-11864-2
Y. Zhang, X. Liang, D. Zhang, M. Tan and E. P. Xing, Unsupervised object-level video summarization with online motion auto-encoder. Pattern Recognition Letters, vol. 130, pp. 376-385, 2020. doi: 10.1016/j.patrec.2018.07.030
V. Tiwari and C. Bhatnagar, A survey of recent work on video summarization: approaches and techniques. Multimedia Tools and Applications, vol. 80, no. 18, pp. 27187-27221, 2021. doi: 10.1007/s11042-021-10977-y
P. Pareek and A. Thakkar, A survey on video-based human action recognition: recent updates, datasets, challenges, and applications. Artificial Intelligence Review, vol. 54, no. 3, pp. 2259-2322, 2021. doi: 10.1007/s10462-020-09904-8
J. Stjepandić and M. Sommer, Object recognition methods in a built environment. DigiTwin: An Approach for Production Process Optimization in a Built Environment, pp. 103-134, 2022. doi: 10.1007/978-3-030-77539-1_6
N. O’Mahony, S. Campbell, A. Carvalho, S. Harapanahalli, G. V. Hernandez, L. Krpalkova, D. Riordan and J. Walsh, Deep learning vs. traditional computer vision. In Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1, no. 1, pp. 128-144, 2020. doi: 10.1007/978-3-030-17795-9_10
C. B. Murthy, M. F. Hashmi, N. D. Bokde and Z. W. Geem. Investigations of object detection in images/videos using various deep learning techniques and embedded platforms—A comprehensive review. Applied sciences, vol. 10, no. 9, pp. 3280, 2020. doi: 10.3390/app10093280
M. Chen, J. Sun, K. Aida and A. Takefusa, Weather-aware object detection method for maritime surveillance systems. Future Generation Computer Systems, vol. 151, pp. 111123, 2024. doi: 10.1016/j.future.2023.09.030
H. Khan, T. Hussain, S. U. Khan, Z. A. Khan and S. W. Baik, Deep multi-scale pyramidal features network for supervised video summarization. Expert Systems with Applications, vol. 237, pp. 121288, 2024. doi: 10.1016/j.eswa.2023.121288
Y. Zhou, A YOLO-NL object detector for real-time detection. Expert Systems with Applications, vol. 238, pp. 122256, 2024. doi: 10.1016/j.eswa.2023.122256
Z. Yunzuo and Z. Tingting, Object interaction-based surveillance video synopsis. Applied Intelligence, vol. 53, no. 4, pp. 4648-4664, 2023. doi: 10.1007/s10489-022-03477-5
P. G. Shambharkar and R. Goel, From video summarization to real time video summarization in smart cities and beyond: A survey. Frontiers in big Data, vol. 5, pp. 1106776, 2023. doi: 10.3389/fdata.2022.1106776
A. Sabha and A. Selwal, Data-driven enabled approaches for criteria-based video summarization: a comprehensive survey, taxonomy, and future directions. Multimedia Tools and Applications, vol, 82, no. 21, pp. 32635-32709, 2023. doi: 10.1007/s11042-02314925-w
M. S. Nair and J. Mohan, Static video summarization using multi-CNN with sparse autoencoder and random forest classifier. Signal, Image and Video Processing, vol. 15, no. 4, pp. 735-742, 2021. doi: 10.1007/s11760-020-01791-4
A. Sabha and A. Selwal, CoSumNet: A video summarization-based framework for COVID-19 monitoring in crowded scenes. Artificial Intelligence in Medicine, vol. 139, pp. 102544, 2023. doi: 10.1016/j.artmed.2023.102544
W. Abdullah, A. Tolba, A. Elmasry and N. N. Mostafa, VisionCam: A Comprehensive XAI Toolkit for Interpreting Image-Based Deep Learning Models. Sustainable Machine Intelligence Journal, vol. 8, pp. 4-46, 2024. doi: 10.61356/SMIJ.2024.8290
A. M. Ali and A. Abdelhafeez, DeepHAR-Net: a novel machine intelligence approach for human activity recognition from inertial sensors. Sustainable Machine Intelligence Journal, vol. 1, pp. 1-1, 2022. doi: 10.61185/SMIJ.2022.8463
A. J. Alkhatib, M. Alharoun, A. Alzoubi, E. Muqdadi and A. A. Aqoulah, Diagnosing Brain Tumors from MRI images through a Multi-Fused CNN with Auxiliary Layers. Sustainable Machine Intelligence Journal, vol. 6, pp. 2-1, 2024. doi: 10.61356/SMIJ.2024.66102
M. Lincy Jacquline and N. Sudha, weighted fuzzy C means and enhanced adaptive neurofuzzy inference based chronic kidney disease classification. Journal of Fuzzy Extension and Applications, vol. 5, no. 1, pp. 100-115, 2024. doi: 10.22105/jfea.2024.437690.1376
S. A. Nourkhah, G. Cirovic and S. A. Edalatpanah, The role of sensors in smart agriculture. Computational Algorithms and Numerical Dimensions, vol. 2, no. 4, pp. 210-215, 2023.
I. Mekawy, Object Detection by Neural Network for Smart Home. Big Data and Computing Visions, vol. 2, 10.22105/bdcv.2022.333756.1072 no. 4, pp. 143-148, 2022. doi:
S. Khajehkhasan and S. Fakheri, A new method based on operational strategies for early detection of breast cancers. Innovation Management and Operational Strategies, vol. 1, no. 2, pp. 187-201, 2020. doi: 10.22105/imos.2021.266543.1027
A. Alzoubi, A. Alaiad, K. Alkhattib, A. J. Alkhatib, A. A. Aqoulah and O. Hayajnah, Detection of Depression from Arabic Tweets Using Machine Learning. Sustainable Machine Intelligence Journal, vol. 6, pp. 3-1, 2024. doi: 10.61356/SMIJ.2024.11103
T. Wagner, A. Gepperth and E. Engels, A framework for the automated parameterization of a sensorless bearing fault detection pipeline, 2023. arXiv preprint arXiv:2303.08858. doi: 10.22105/jarie.2023.391005.1538
A. G. Talouki, A. Koochari and S. A. Edalatpanah, Image completion based on segmentation using neutrosophic sets. Expert systems with applications, vol. 238, pp. 121769, 2024. doi: 10.1016/j.eswa.2023.121769
M. Dirik, Fire extinguishers based on acoustic oscillations in airflow using fuzzy classification. Journal of fuzzy extension and applications, vol. 4, no. 3, pp. 217-234, 2023. doi: 10.22105/jfea.2023.401426.1291
A. Baig, Deep attributes and decisions fusion for no-reference video quality analysis. Big Data and Computing Visions, vol. 3, no. 3, pp. 91-103, 2023, doi 10.22105/bdcv.2023.415895.1165
https://www.youtube.com/watch?v=4zfVFFw5nOg
A. Karbalaie, F. Abtahi and M. Sjöström, Event detection in surveillance videos: a review. Multimedia tools and applications, vol. 81, no. 24, pp. 35463-35501, 2022. doi: 10.1007/s11042-021-11864-2
Y. Zhang, X. Liang, D. Zhang, M. Tan and E. P. Xing, Unsupervised object-level video summarization with online motion auto-encoder. Pattern Recognition Letters, vol. 130, pp. 376-385, 2020. doi: 10.1016/j.patrec.2018.07.030
V. Tiwari and C. Bhatnagar, A survey of recent work on video summarization: approaches and techniques. Multimedia Tools and Applications, vol. 80, no. 18, pp. 27187-27221, 2021. doi: 10.1007/s11042-021-10977-y
P. Pareek and A. Thakkar, A survey on video-based human action recognition: recent updates, datasets, challenges, and applications. Artificial Intelligence Review, vol. 54, no. 3, pp. 2259-2322, 2021. doi: 10.1007/s10462-020-09904-8
J. Stjepandić and M. Sommer, Object recognition methods in a built environment. DigiTwin: An Approach for Production Process Optimization in a Built Environment, pp. 103-134, 2022. doi: 10.1007/978-3-030-77539-1_6
N. O’Mahony, S. Campbell, A. Carvalho, S. Harapanahalli, G. V. Hernandez, L. Krpalkova, D. Riordan and J. Walsh, Deep learning vs. traditional computer vision. In Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1, no. 1, pp. 128-144, 2020. doi: 10.1007/978-3-030-17795-9_10
C. B. Murthy, M. F. Hashmi, N. D. Bokde and Z. W. Geem. Investigations of object detection in images/videos using various deep learning techniques and embedded platforms—A comprehensive review. Applied sciences, vol. 10, no. 9, pp. 3280, 2020. doi: 10.3390/app10093280
M. Chen, J. Sun, K. Aida and A. Takefusa, Weather-aware object detection method for maritime surveillance systems. Future Generation Computer Systems, vol. 151, pp. 111123, 2024. doi: 10.1016/j.future.2023.09.030
H. Khan, T. Hussain, S. U. Khan, Z. A. Khan and S. W. Baik, Deep multi-scale pyramidal features network for supervised video summarization. Expert Systems with Applications, vol. 237, pp. 121288, 2024. doi: 10.1016/j.eswa.2023.121288
Y. Zhou, A YOLO-NL object detector for real-time detection. Expert Systems with Applications, vol. 238, pp. 122256, 2024. doi: 10.1016/j.eswa.2023.122256
Z. Yunzuo and Z. Tingting, Object interaction-based surveillance video synopsis. Applied Intelligence, vol. 53, no. 4, pp. 4648-4664, 2023. doi: 10.1007/s10489-022-03477-5
P. G. Shambharkar and R. Goel, From video summarization to real time video summarization in smart cities and beyond: A survey. Frontiers in big Data, vol. 5, pp. 1106776, 2023. doi: 10.3389/fdata.2022.1106776
A. Sabha and A. Selwal, Data-driven enabled approaches for criteria-based video summarization: a comprehensive survey, taxonomy, and future directions. Multimedia Tools and Applications, vol, 82, no. 21, pp. 32635-32709, 2023. doi: 10.1007/s11042-02314925-w
M. S. Nair and J. Mohan, Static video summarization using multi-CNN with sparse autoencoder and random forest classifier. Signal, Image and Video Processing, vol. 15, no. 4, pp. 735-742, 2021. doi: 10.1007/s11760-020-01791-4
A. Sabha and A. Selwal, CoSumNet: A video summarization-based framework for COVID-19 monitoring in crowded scenes. Artificial Intelligence in Medicine, vol. 139, pp. 102544, 2023. doi: 10.1016/j.artmed.2023.102544
W. Abdullah, A. Tolba, A. Elmasry and N. N. Mostafa, VisionCam: A Comprehensive XAI Toolkit for Interpreting Image-Based Deep Learning Models. Sustainable Machine Intelligence Journal, vol. 8, pp. 4-46, 2024. doi: 10.61356/SMIJ.2024.8290
A. M. Ali and A. Abdelhafeez, DeepHAR-Net: a novel machine intelligence approach for human activity recognition from inertial sensors. Sustainable Machine Intelligence Journal, vol. 1, pp. 1-1, 2022. doi: 10.61185/SMIJ.2022.8463
A. J. Alkhatib, M. Alharoun, A. Alzoubi, E. Muqdadi and A. A. Aqoulah, Diagnosing Brain Tumors from MRI images through a Multi-Fused CNN with Auxiliary Layers. Sustainable Machine Intelligence Journal, vol. 6, pp. 2-1, 2024. doi: 10.61356/SMIJ.2024.66102
M. Lincy Jacquline and N. Sudha, weighted fuzzy C means and enhanced adaptive neurofuzzy inference based chronic kidney disease classification. Journal of Fuzzy Extension and Applications, vol. 5, no. 1, pp. 100-115, 2024. doi: 10.22105/jfea.2024.437690.1376
S. A. Nourkhah, G. Cirovic and S. A. Edalatpanah, The role of sensors in smart agriculture. Computational Algorithms and Numerical Dimensions, vol. 2, no. 4, pp. 210-215, 2023.
I. Mekawy, Object Detection by Neural Network for Smart Home. Big Data and Computing Visions, vol. 2, 10.22105/bdcv.2022.333756.1072 no. 4, pp. 143-148, 2022. doi:
S. Khajehkhasan and S. Fakheri, A new method based on operational strategies for early detection of breast cancers. Innovation Management and Operational Strategies, vol. 1, no. 2, pp. 187-201, 2020. doi: 10.22105/imos.2021.266543.1027
A. Alzoubi, A. Alaiad, K. Alkhattib, A. J. Alkhatib, A. A. Aqoulah and O. Hayajnah, Detection of Depression from Arabic Tweets Using Machine Learning. Sustainable Machine Intelligence Journal, vol. 6, pp. 3-1, 2024. doi: 10.61356/SMIJ.2024.11103
T. Wagner, A. Gepperth and E. Engels, A framework for the automated parameterization of a sensorless bearing fault detection pipeline, 2023. arXiv preprint arXiv:2303.08858. doi: 10.22105/jarie.2023.391005.1538
A. G. Talouki, A. Koochari and S. A. Edalatpanah, Image completion based on segmentation using neutrosophic sets. Expert systems with applications, vol. 238, pp. 121769, 2024. doi: 10.1016/j.eswa.2023.121769
M. Dirik, Fire extinguishers based on acoustic oscillations in airflow using fuzzy classification. Journal of fuzzy extension and applications, vol. 4, no. 3, pp. 217-234, 2023. doi: 10.22105/jfea.2023.401426.1291
A. Baig, Deep attributes and decisions fusion for no-reference video quality analysis. Big Data and Computing Visions, vol. 3, no. 3, pp. 91-103, 2023, doi 10.22105/bdcv.2023.415895.1165
https://www.youtube.com/watch?v=4zfVFFw5nOg
Published
2025-04-08
How to Cite
INAYATHULLA, Mohammed; C, Karthikeyan.
Object Detection in Video Summarization for Video Surveillance Applications.
Yugoslav Journal of Operations Research, [S.l.], apr. 2025.
ISSN 2334-6043.
Available at: <https://yujor.fon.bg.ac.rs/index.php/yujor/article/view/1332>. Date accessed: 14 apr. 2025.
doi: https://doi.org/10.2298/YJOR240615010I.
Issue
Section
Research Articles
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
