You Only Look Once v5 and Long Short-Term Memory Implementation for Crowd Anomaly Detection
DOI:
https://doi.org/10.52436/1.jutif.2025.6.2.4224Keywords:
Crowd Anomaly Detection, Long Short-Term Memory, Surveillance systems, You Only Look Once v5Abstract
In Indonesia, 116,000 traffic accidents and 370,747 workplace accidents occurred in 2023, emphasizing the urgent need for effective surveillance systems for monitoring crowded areas such as public sidewalks, roads, workplaces, and school hallways. This study introduces a novel approach combining You Only Look Once v5 (YOLOv5) and Long Short-Term Memory (LSTM) networks for crowd anomaly detection. Unlike traditional methods, this hybrid framework utilizes YOLOv5 for precise feature extraction from video frames and LSTM to capture temporal dependencies for detecting anomalous behaviors. The dataset used includes scenes from the Crowd Anomaly Detection UML Dataset, consisting of a 1-minute and 11-second video extracted into 852 images. Hyperparameter tuning was conducted for epochs and learning rates in the YOLOv5 model, as well as for epochs and units in the LSTM model. The proposed framework achieved remarkable results, with 98% accuracy, 100% precision, and 86% F1-Score. However, improvements in class distribution within the training data could enhance model performance further. These findings demonstrate the potential of the proposed method for real-world applications in improving public safety and effective anomaly detection. This research proves that the proposed method which uses separate feature extraction method before detecting anomaly provides a better result in crowd anomaly detection.
Downloads
References
Fernández-Ares, P. García-Sánchez, M. G. Arenas, A. M. Mora and P. A. CastilloValdivieso, "Detection and Analysis of Anomalies in People Density and Mobility Through Wireless Smartphone Tracking," IEEE Access, vol. 8, pp. 54237-54253, 2020, doi: 10.1109/ACCESS.2020.2979367.
T. Kulshrestha, D. Saxena, R. Niyogi and J. Cao, "Real-Time Crowd Monitoring Using Seamless Indoor-Outdoor Localization," IEEE Transactions on Mobile Computing, vol. 19, no. 3, pp. 664-679, 2020, doi: 10.1109/TMC.2019.2897561.
E. I. Sari, " Angka Kematian Kecelakaan Transportasi Turun Sepanjang 2023." 2023. https://indonesia.go.id/kategori/editorial/7879/angka-kematian kecelakaantransportasi-turun-sepanjang-2023 (accessed Mar. 11, 2024).
Kementerian Ketenagakerjaan Republik Indonesia, "Kecelakaan kerja di Indonesia Tahun 2023." 2023. https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fsatudata.kemnaker.go.id%2Fsatudatapublic%2F2023%2F11%2Ffiles%2Fdata%2F1708925833582_Jumlah%252520kecelakaan%252520kerja%252520tahun%2525202023.xlsx&wdOrigin=BROWSELINK (accessed Mar. 12, 2024).
X. Zhang, J. Fang, B. Yang, S. Chen and B. Li, "Hybrid Attention and Motion Constraint for Anomaly Detection in Crowded Scenes," IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 5, pp. 2259-2274, 2023, doi: 10.1109/TCSVT.2022.3221622.
Z. Ilyas, Z. Aziz, T. Qasim, et al., "A hybrid deep network based approach for crowd anomaly detection," Multimedia Tools and Applications, vol. 80, pp. 24053–24067, 2021, doi: 10.1007/s11042-021-10785-4.
C. Direkoglu, “Abnormal Crowd Behavior Detection Using Motion Information Images and Convolutional Neural Networks,” IEEE Access, vol. 8, pp. 80408–80416, 2020, doi: 10.1109/ACCESS.2020.2990355.
F. Abdullah, M. Abdelhaq, R. Alsaqour, M. H. Alatiyyah, K. Alnowaiser, S. S. Alotaibi, et al., "Context Aware Crowd Tracking and Anomaly Detection via Deep Learning and Social Force Model," IEEE Access, vol. 11, pp. 75884-75898, 2023, doi: 10.1109/ACCESS.2023.3293537.
A. A. Khan, M. A. Nauman, M. Shoaib, R. Jahangir, R. Alroobaea, M. Alsafyani, et al., "Crowd Anomaly Detection in Video Frames Using Fine-Tuned AlexNet Model," Electronics, vol. 11, no. 19, p. 3105, 2022, doi: 10.3390/electronics11193105.
K. Pawar and V. Attar, "Application of Deep Learning for Crowd Anomaly Detection from Surveillance Videos," in 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 506-511, 2021, doi: 10.1109/Confluence51648.2021.9377055.
C.-W. Chang, C. Chang, and Y.-Y. Lin, “A hybrid CNN and LSTM-based deep learning model for abnormal behavior detection,” Multimedia Tools and Applications, vol. 81, pp. 11825–11843, Jan. 2022, doi: 10.1007/s11042-021-11887-9.
A. Arif and A. Jalal, ‘‘Automated body parts estimation and detection using salient maps and Gaussian matrix model,’’ in Proc. Int. Bhurban Conf. Appl. Sci. Technol. (IBCAST), 2021, pp. 667–672, doi: 10.1109/IBCAST51254.2021.9393268.
K. Rezaee, S. M. Rezakhani, M. R. Khosravi, and M. K. Moghimi, ‘‘A survey on deep learning-based real-time crowd anomaly detection for secure distributed video surveillance,’’ Pers. Ubiquitous Comput., 2021, vol. 28, pp. 135–151, doi: 10.1007/s00779-021-01586-5.
K. Boekhoudt, A. Matei, M. Aghaei, and E. Talavera, ‘‘HR-Crime: Humanrelated anomaly detection in surveillance videos,’’ in Proc. CAIP. Cham, Switzerland: Springer, 2021, vol. 13053, pp. 164–174, doi: 10.1007/978-3-030-89131-2_15.
L. Xia and Z. Li, "A new method of abnormal behavior detection using LSTM network with temporal attention mechanism," The Journal of Supercomputing, vol. 77, no. 4, pp. 3223–3241, 2021, doi: 10.1007/s11227-020-03391-y.
W. Luo, W. Liu, D. Lian, and S. Gao, "Video anomaly detection with sparse coding inspired deep neural networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 3, pp. 1070–1084, 2021, doi: 10.1109/TPAMI.2019.2944377.
X. Li, M. Chen and Q. Wang, "Quantifying and Detecting Collective Motion in Crowd Scenes," IEEE Transactions on Image Processing, vol. 29, pp. 5571-5583, 2020, doi: 10.1109/TIP.2020.2985284.
A. Bamaqa, M. Sedky, and B. Bastaki, "Reactive and Proactive Anomaly Detection in Crowd Management Using Hierarchical Temporal Memory," International Journal of Machine Learning and Computing, vol. 12, no. 1, pp. 7-16, 2022, doi: 10.18178/ijmlc.2022.12.1.1072.
H. Sivalingan and N. Anandakrishnan, "Crowd Localization and Anomaly Detection Using Video Anomaly Scoring Network," MSEA, vol. 72, no. 1, pp. 825–837, Mar. 2023, doi: 10.17762/msea.v72i1.2055.
A. Mehmood, "Efficient Anomaly Detection in Crowd Videos Using Pre-Trained 2D Convolutional Neural Networks," IEEE Access, vol. 9, pp. 138283-138295, 2021, doi: 10.1109/ACCESS.2021.3118009.
J. Li, Q. Huang, Y. Du, X. Zhen, S. Chen and L. Shao, "Variational Abnormal Behavior Detection With Motion Consistency," IEEE Transactions on Image Processing, vol.31, pp. 275-286, 2022, doi: 10.1109/TIP.2021.3130545.
A. Pelati, M. Meo and P. Dini, "Traffic Anomaly Detection Using Deep SemiSupervised Learning at the Mobile Edge," IEEE Transactions on Vehicular Technology, vol. 71, no. 8, pp. 8919-8932, Aug. 2022, doi: 10.1109/TVT.2022.3174735.
M. Sabih and D. K. Vishwakarma, "Crowd anomaly detection with LSTMs using optical features and domain knowledge for improved inferring," The Visual Computer, vol. 38, pp. 1719–1730, 2022, doi: 10.1007/s00371-021-02100-x.
M. Ş. Gündüz and G. Işık, "A new YOLO-based method for real-time crowd detection from video and performance analysis of YOLO models," Journal of Real-Time Image Processing, vol. 20, no. 5, pp. 1239–1254, 2023, doi: 10.1007/s11554-023-01276-w.
T. Saba, "Real-time anomalies detection in crowd using convolutional long short-term memory network," Journal of Information Science, vol. 49, no. 5, pp. 1145-1152, 2023, doi: 10.1177/01655515211022665.
G. Jocher, ”Ultralytics YOLOv5 Architecture,” 2023. https://docs.ultralytics.com/yolov5/tutorials/architecture_description/ (accessed Mar. 23, 2024).
H. Liu, F. Sun, J. Gu, and L. Deng, "SF-YOLOv5: A Lightweight Small Object Detection Algorithm Based on Improved Feature Fusion Mode," Sensors, vol. 22, no. 15, pp. 5817, Aug. 2022, doi: 10.3390/s22155817.
A.N. Moustafa and W. Gomaa, "Gate and common pathway detection in crowd scenes and anomaly detection using motion units and LSTM predictive models," Multimed Tools Appl., vol. 79, pp. 20689–20728, 2020. doi: 10.1007/s11042-020-08840-7.
P. Rivas, Deep Learning for Beginners: A beginner’s guide to getting up and running with deep learning from scratch using Python. Packt Publishing Ltd.
I. P. Sary, S. Andromeda, and E. U. Armin, “Performance Comparison of YOLOv5 and YOLOv8 Architectures in Human Detection using Aerial Images,” Ultima Computing: Jurnal Sistem Komputer, vol. 15, no. 1, pp. 8– 13, 2023, doi: 10.31937/sk.v15i1.3204.
U. Nepal and H. Eslamiat, “Comparing YOLOv3, YOLOv4 and YOLOv5 for Autonomous Landing Spot Detection in Faulty UAVs,” Sensors, vol. 22, no. 2, 2022, doi: 10.3390/s22020464.
K. Khairunnas, E. M. Yuniarno, and A. Zaini, “Pembuatan Modul Deteksi Objek Manusia Menggunakan Metode YOLO untuk Mobile Robot,” Jurnal Teknik ITS, vol. 10, no. 1, pp. A50–A55, 2021, doi: 10.12962/j23373539.v10i1.61622.
E. Casas, L. Ramos, E. Bendek, and F. RivasEcheverria, “Assessing the Effectiveness of YOLO Architectures for Smoke and Wildfire Detection,” IEEE Access, vol. 11, pp. 96554– 96583, 2023, doi: 10.1109/ACCESS.2023.3312217.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Ken Ratri Retno Wardani, Nicholas Chrisandy, Inge Martina, Hery Heryanto
This work is licensed under a Creative Commons Attribution 4.0 International License.
