Enhancing Cyberbullying Detection with a CNN-GRU Hybrid Model, Word2Vec, and Attention Mechanism
DOI:
https://doi.org/10.52436/1.jutif.2025.6.3.4176Keywords:
Attention Mechanism, Cyberbullying Detection, Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), Word2VecAbstract
Cyberbullying is an act of violence commonly committed on online platforms such as social media X, often causing psychological effects for victims. Despite prevention efforts, traditional methods for detecting cyberbullying show limited effectiveness due to the complexity of language and diversity of expressions, leading to suboptimal performance. This study aims to enhance detection accuracy by applying Convolutional Neural Networks (CNN) and Gated Recurrent Unit (GRU) with an attention mechanism to analyze textual data from tweets. The model uses Term Frequency-Inverse Document Frequency (TF-IDF) for extracting important words and Word2Vec for expanding text representation. A total of 30,084 labeled datasets from tweets on social media X were utilized. Results indicate the hybrid CNN-GRU model with attention achieved the highest accuracy of 80.96%, outperforming stand-alone CNN and GRU models. Additionally, TF-IDF and Word2Vec significantly improved model performance, with the CNN-GRU combination proving most effective for detecting cyberbullying. This study contributes to computer science by proposing a novel approach that integrates CNN, GRU, and attention mechanisms with advanced feature extraction techniques, providing a more reliable detection system for online platforms. It also highlights the potential for integrating multimodal data to further enhance future performance.
Downloads
References
N. T. Martoredjo, “Social media as a learning tool in the digital age: A review,” in International Conference on Computer Science and Computational Intelligence, 2023, pp. 534–539. doi: 10.1016/j.procs.2023.10.555.
“X/Twitter: Countries with the largest audience 2024,” Statista Research Department. Accessed: Dec. 04, 2024. [Online]. Available: https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/
G. K. Nathanael, “Understanding recession response by Twitter users: A text analysis approach,” Heliyon, vol. 10, no. 1, Jan. 2024, doi: 10.1016/j.heliyon.2023.e23737.
K. Hellfeldt, L. López-Romero, and H. Andershed, “Cyberbullying and psychological well-being in young adolescence: the potential protective mediation effects of social support from family, friends, and teachers,” Int J Environ Res Public Health, vol. 17, no. 1, Jan. 2020, doi: 10.3390/ijerph17010045.
N. W. F. Amalia, “Cyberbullying Detection on Twitter using Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU),” Telkom University, 2023.
Y. Setiawan, N. U. Maulidevi, and K. Surendro, “The Use of Dynamic n-Gram to Enhance TF-IDF Features Extraction for Bahasa Indonesia Cyberbullying Classification,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Feb. 2023, pp. 200–205. doi: 10.1145/3587828.3587858.
L. Ye, C. Wei, N. Heran, and Y. Yimeng, “Review Mining for Experiential Products Incorporating Word2vec and Review Sentiment Tendencies,” in Procedia Computer Science, Elsevier B.V., 2022, pp. 1492–1499. doi: 10.1016/j.procs.2022.11.335.
A. Aljohani, N. Alharbe, R. E. Al Mamlook, and M. M. Khayyat, “A hybrid combination of CNN Attention with optimized random forest with grey wolf optimizer to discriminate between Arabic hateful, abusive tweets,” Journal of King Saud University - Computer and Information Sciences, vol. 36, no. 2, Feb. 2024, doi: 10.1016/j.jksuci.2024.101961.
Q. Liu, Y. Hu, and H. Liu, “Enhanced stock price prediction with optimized ensemble modeling using multi-source heterogeneous data: Integrating LSTM attention mechanism and multidimensional gray model,” J Ind Inf Integr, vol. 42, Nov. 2024, doi: 10.1016/j.jii.2024.100711.
Gaurav and P. Mathur, “An Attention Mechanism and GRU Based Deep Learning Model for Automatic Image Captioning,” International Journal of Engineering Trends and Technology, vol. 70, no. 3, pp. 302–309, Mar. 2022, doi: 10.14445/22315381/IJETT-V70I3P234.
A. Perera and P. Fernando, “Accurate cyberbullying detection and prevention on social media,” in CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies, 2020, pp. 605–611. doi: 10.1016/j.procs.2021.01.207.
M. M. Abedi and E. Sacchi, “A machine learning tool for collecting and analyzing subjective road safety data from Twitter,” Expert Syst Appl, vol. 240, Apr. 2024, doi: 10.1016/j.eswa.2023.122582.
J. Zhou, Z. Ye, S. Zhang, Z. Geng, N. Han, and T. Yang, “Investigating response behavior through TF-IDF and Word2vec text analysis: A case study of PISA 2012 problem-solving process data,” Heliyon, vol. 10, no. 16, Aug. 2024, doi: 10.1016/j.heliyon.2024.e35945.
M. Deja, Isto Huvila, G. Widén, and F. Ahmad, “Seeking innovation: The research protocol for SMEs’ networking,” Heliyon, vol. 9, no. 4, Apr. 2023, doi: 10.1016/j.heliyon.2023.e14689.
R. Agrawal and R. Goyal, “Developing bug severity prediction models using word2vec,” International Journal of Cognitive Computing in Engineering, vol. 2, pp. 104–115, Jun. 2021, doi: 10.1016/j.ijcce.2021.08.001.
H. Imaduddin and S. Fauziati, “Word Embedding Comparison for Indonesian Language Sentiment Analysis,” in International Conference of Artificial Intelligence and Information Technology, 2019, pp. 426–430. doi: 10.1109/ICAIIT.2019.8834536.
L. Shan, Y. Liu, M. Tang, M. Yang, and X. Bai, “CNN-BiLSTM hybrid neural networks with attention mechanism for well log prediction,” J Pet Sci Eng, vol. 205, Oct. 2021, doi: 10.1016/j.petrol.2021.108838.
M. N. I. Siddique, M. Shafiullah, S. Mekhilef, H. Pota, and M. A. Abido, “Fault classification and location of a PMU-equipped active distribution network using deep convolution neural network (CNN),” Electric Power Systems Research, vol. 229, Apr. 2024, doi: 10.1016/j.epsr.2024.110178.
Y. Zhang and H. D. Fill, “TS-GRU: A Stock Gated Recurrent Unit Model Driven via Neuro-Inspired Computation,” Electronics (Basel), vol. 13, no. 23, p. 4659, Nov. 2024, doi: 10.3390/electronics13234659.
T. Li, Y. Lin, B. Cheng, G. Ai, J. Yang, and L. Fang, “PU-CTG: A Point Cloud Upsampling Network Using Transformer Fusion and GRU Correction,” Remote Sens (Basel), vol. 16, no. 3, Feb. 2024, doi: 10.3390/rs16030450.
Y. Yan et al., “Hybrid GRU–Random Forest Model for Accurate Atmospheric Duct Detection with Incomplete Sounding Data,” Remote Sens (Basel), vol. 16, no. 22, Nov. 2024, doi: 10.3390/rs16224308.
A. Zhang, S. Chun, Z. Cheng, and P. Zhao, “Predicting the core thermal hydraulic parameters with a gated recurrent unit model based on the soft attention mechanism,” Nuclear Engineering and Technology, Mar. 2024, doi: 10.1016/j.net.2024.01.045.
W. Jia, Y. Zhan, J. Zhang, and Y. Dai, “Robot assisted bone milling state classification network with attention mechanism,” Expert Syst Appl, vol. 249, Sep. 2024, doi: 10.1016/j.eswa.2024.123726.
E. Lieskovská, M. Jakubec, R. Jarina, and M. Chmulík, “A review on speech emotion recognition using deep learning and attention mechanism,” May 02, 2021, MDPI AG. doi: 10.3390/electronics10101163.
G. Brauwers and F. Frasincar, “A General Survey on Attention Mechanisms in Deep Learning,” IEEE Trans Knowl Data Eng, vol. 35, no. 4, pp. 3279–3298, Apr. 2023, doi: 10.1109/TKDE.2021.3126456.
D. Soydaner, “Attention mechanism in neural networks: where it comes and where it goes,” Aug. 01, 2022, Springer Science and Business Media Deutschland GmbH. doi: 10.1007/s00521-022-07366-3.
X. Deng, Q. Liu, Y. Deng, and S. Mahadevan, “An improved method to construct basic probability assignment based on the confusion matrix for classification problem,” Inf Sci (N Y), vol. 340–341, pp. 250–261, May 2016, doi: 10.1016/j.ins.2016.01.033.
“Confusion Matrix.” Accessed: Apr. 22, 2024. [Online]. Available: https://www.sciencedirect.com/topics/engineering/confusion-matrix
G. Phillips et al., “Setting nutrient boundaries to protect aquatic communities: The importance of comparing observed and predicted classifications using measures derived from a confusion matrix,” Science of the Total Environment, vol. 912, Feb. 2024, doi: 10.1016/j.scitotenv.2023.168872.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Kaysa Azzahra Adriana, Erwin Budi Setiawan
This work is licensed under a Creative Commons Attribution 4.0 International License.
