Depression Detection using Convolutional Neural Networks and Bidirectional Long Short-Term Memory with BERT variations and FastText Methods
DOI:
https://doi.org/10.52436/1.jutif.2025.6.3.4874Keywords:
BERT, BiLSTM, CNN, Depression detection, FastText, Social MediaAbstract
Depression has become a significant public health concern in Indonesia, with many individuals expressing mental distress through social media platforms like Twitter. As mental health issues like depression are increasingly prevalent in the digital age, social media provides a valuable avenue for automated detection via text, though obstacles such as informal language, vagueness, and contextual complexity in social media complicate precise identification. This study aims to develop an effective depression detection model using Indonesian tweets by combining Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM). The dataset consisted of 58,115 tweets, labeled into depressed and non-depressed categories. The data were preprocessed, followed by feature extraction using BERT and feature expansion using FastText. The FastText model was trained on three corpora: Tweet, IndoNews, and combined Tweet+IndoNews corpus; the total corpus will be 169,564 entries. The best result was achieved by BiLSTM model with 84.67% accuracy, a 1.94% increase from the baseline, and the second best was the BiLSTM-CNN hybrid model achieved 84.61 with an accuracy increase of 1.7% from the baseline. These result indicate that combining semantic feature expansion with deep learning architecture effectively improves the accuracy of depression detection on social media platforms. These insights highlight the importance of integrating semantic enrichment and contextual modeling to advance automated mental health diagnostics in Indonesian digital ecosystems.
Downloads
References
Vandana, N. Marriwala, and D. Chaudhary, “A hybrid model for depression detection using deep learning,” Meas. Sensors, vol. 25, no. December 2022, p. 100587, 2023, doi: 10.1016/j.measen.2022.100587.
Z. Wang, L. Chen, L. Wang, and G. Diao, “Recognition of Audio Depression Based on Convolutional Neural Network and Generative Antagonism Network Model,” IEEE Access, vol. 8, pp. 101181–101191, 2020, doi: 10.1109/ACCESS.2020.2998532.
G. Rao, Y. Zhang, L. Zhang, Q. Cong, and Z. Feng, “MGL-CNN: A Hierarchical Posts Representations Model for Identifying Depressed Individuals in Online Forums,” IEEE Access, vol. 8, pp. 32395–32403, 2020, doi: 10.1109/ACCESS.2020.2973737.
X. Zhang et al., “The influence of genetic and acquired factors on the vulnerability to develop depression: a review,” Biosci. Rep., vol. 43, no. 5, pp. 1–17, 2023, doi: 10.1042/BSR20222644.
World Health Organization, “Depressive disorder (depression),” Depressive disorder (depression). Accessed: Apr. 29, 2025. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/depression
C. G. Davey and P. D. Mcgorry, “Early intervention for depression in young people : a blind spot in mental health care,” The Lancet Psychiatry, vol. 0366, no. 18, 2018, doi: 10.1016/S2215-0366(18)30292-X.
Y. Chen et al., “Effectiveness of health checkup with depression screening on depression treatment and outcomes in middle-aged and older adults : a target trial emulation study,” Lancet Reg. Heal. - West. Pacific, vol. 43, p. 100978, 2024, doi: 10.1016/j.lanwpc.2023.100978.
A. A. Jamali, C. Berger, and R. J. Spiteri, “Momentary Depressive Feeling Detection Using X (Formerly Twitter) Data: Contextual Language Approach,” Jmir Ai, vol. 2, p. e49531, 2023, doi: 10.2196/49531.
B. G. Bokolo and Q. Liu, “Deep Learning-Based Depression Detection from Social Media: Comparative Evaluation of ML and Transformer Techniques,” Electron., vol. 12, no. 21, 2023, doi: 10.3390/electronics12214396.
K. C. Bathina, L. Lorenzo-luaces, L. A. Rutter, B. Complexity, and B. Sciences, “Depressed individuals express more distorted thinking on social media,” no. Cd, pp. 1–21, 2020, doi: https://doi.org/10.48550/arXiv.2002.02800.
A. Ranjith Kumar, K. Aditya, S. Antony Joseph Raj, and V. Nandhakumar, “Depression Detection Using Optical Characteristic Recognition and Natural Language Processing in SNS,” 4th Int. Conf. Comput. Commun. Signal Process. ICCCSP 2020, 2020, doi: 10.1109/ICCCSP49186.2020.9315254.
D. Liu, X. L. Feng, F. Ahmed, M. Shahid, and J. Guo, “Detecting and Measuring Depression on Social Media Using a Machine Learning Approach: Systematic Review,” JMIR Ment. Heal., vol. 9, no. 3, pp. 1–18, 2022, doi: 10.2196/27244.
H. Kour and M. K. Gupta, “An hybrid deep learning approach for depression prediction from user tweets using feature-rich CNN and bi-directional LSTM,” Multimed. Tools Appl., vol. 81, no. 17, 2022, doi: 10.1007/s11042-022-12648-y.
J. Choi and S. W. Lee, “Improving FastText with inverse document frequency of subwords,” Pattern Recognit. Lett., vol. 133, pp. 165–172, 2020, doi: 10.1016/j.patrec.2020.03.003.
N. Sabharwal and A. Agrawal, “Introduction to Word Embeddings,” in Hands-on Question Answering Systems with BERT, Apress, 2021, p. 41. doi: 10.1007/978-1-4842-6664-9_3.
M. A. S. Nasution and E. B. Setiawan, “Enhancing Cyberbullying Detection on Indonesian Twitter: Leveraging FastText for Feature Expansion and Hybrid Approach Applying CNN and BiLSTM,” Rev. d’Intelligence Artif., vol. 37, no. 4, pp. 929–936, 2023, doi: 10.18280/ria.370413.
F. Ullah, A. Alsirhani, M. M. Alshahrani, A. Alomari, H. Naeem, and S. A. Shah, “Explainable Malware Detection System Using Transformers-Based Transfer Learning and Multi-Model Visual Representation,” Sensors, vol. 22, no. 18, 2022, doi: 10.3390/s22186766.
S. F. N. Azizah, H. D. Cahyono, S. W. Sihwi, and W. Widiarto, “Performance Analysis of Transformer Based Models (BERT, ALBERT, and RoBERTa) in Fake News Detection,” 2023 6th Int. Conf. Inf. Commun. Technol. ICOIACT 2023, pp. 425–430, 2023, doi: 10.1109/ICOIACT59844.2023.10455849.
I. Tavchioski, M. Robnik-Šikonja, and S. Pollak, “Detection of depression on social networks using transformers and ensembles,” 2023, doi: 10.14746/amup.9788323241775.
B. A. Putri and E. B. Setiawan, “Topic Classification Using the Long Short-Term Memory (LSTM) Method with FastText Feature Expansion on Twitter,” 2023 Int. Conf. Data Sci. Its Appl. ICoDSA 2023, no. April, pp. 18–23, 2023, doi: 10.1109/ICoDSA58501.2023.10277033.
V. Çetin and O. Yıldız, “A comprehensive review on data preprocessing techniques in data analysis,” Pamukkale Univ. J. Eng. Sci., vol. 28, no. 2, pp. 299–312, 2022, doi: 10.5505/pajes.2021.62687.
R. Menon, S. Gide, S. Ghatte, S. Mendon, M. Nashipudimath, and U. G. Student, “Depression Prediction using BERT and SVM,” Int. Res. J. Eng. Technol., pp. 2013–2016, 2021, [Online]. Available: www.irjet.net
D. Cortiz, “Exploring Transformers in Emotion Recognition: a comparison of BERT, DistillBERT, RoBERTa, XLNet and ELECTRA,” pp. 1–7, 2021, [Online]. Available: http://arxiv.org/abs/2104.02041
R. K. Kaliyar, A. Goswami, and P. Narang, “FakeBERT: Fake news detection in social media with a BERT-based deep learning approach,” Multimed. Tools Appl., vol. 80, no. 8, pp. 11765–11788, 2021, doi: 10.1007/s11042-020-10183-2.
K. M. Ang et al., “Optimizing Image Classification: Automated Deep Learning Architecture Crafting with Network and Learning Hyperparameter Tuning,” Biomimetics, vol. 8, no. 7, 2023, doi: 10.3390/biomimetics8070525.
S. Soni, S. S. Chouhan, and S. S. Rathore, “TextConvoNet: a convolutional neural network based architecture for text classification,” Appl. Intell., vol. 53, no. 11, pp. 14249–14268, 2023, doi: 10.1007/s10489-022-04221-9.
P. Nevavuori, N. Narra, P. Linna, and T. Lipping, “Crop yield prediction using multitemporal UAV data and spatio-temporal deep learning models,” Remote Sens., vol. 12, no. 23, pp. 1–18, 2020, doi: 10.3390/rs12234000.
J. Liu, Y. Zhao, Y. Feng, Y. Hu, and X. Ma, “SeMalBERT: Semantic-based malware detection with bidirectional encoder representations from transformers,” J. Inf. Secur. Appl., vol. 80, p. 103690, Feb. 2024, doi: 10.1016/J.JISA.2023.103690.
J. Forry Kusuma and A. Chowanda, “Indonesian Hate Speech Detection Using IndoBERTweet and BiLSTM on Twitter,” Int. J. Informatics Vis., vol. 7, no. September, pp. 773–780, 2023, [Online]. Available: www.joiv.org/index.php/joiv
T. H. H. Aldhyani, S. N. Alsubari, A. S. Alshebami, H. Alkahtani, and Z. A. T. Ahmed, “Detecting and Analyzing Suicidal Ideation on Social Media Using Deep Learning and Machine Learning Models,” Int. J. Environ. Res. Public Health, vol. 19, no. 19, 2022, doi: 10.3390/ijerph191912635.
A. Abdurrahim and D. H. F. Fudholi, “Mental Health Prediction Model on Social Media Data Using CNN-BiLSTM,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, no. 1, 2024, doi: 10.22219/kinetik.v9i1.1849.
W. Li, L. Zhu, Y. Shi, K. Guo, and E. Cambria, “User reviews: Sentiment analysis using lexicon integrated two-channel CNN–LSTM family models,” 2020. doi: 10.1016/j.asoc.2020.106435.
X. Jiao, “TinyBERT : Distilling BERT for Natural Language Understanding,” pp. 4163–4174, 2020.
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT , a distilled version of BERT : smaller , faster , cheaper and lighter,” pp. 2–6, 2019.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, Association for Computational Linguistics (ACL), 2019, pp. 4171–4186.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Leonardus Adi Widjayanto, Erwin Budi Setiawan
This work is licensed under a Creative Commons Attribution 4.0 International License.
