An Evaluation of Self-Attentive Sequential Recommendation (SASRec) Algorithm Using Hyperparameter Tuning

Agung Toto Wibowo; Hasmawati Hasmawati; Hani Nurrahmi; Imtitsal Ulya  Salsabila

doi:10.52436/1.jutif.2026.7.2.5158

Authors

Agung Toto Wibowo School of Computing, Universitas Telkom, Indonesia
Hasmawati School of Computing, Universitas Telkom, Indonesia
Hani Nurrahmi School of Computing, Universitas Telkom, Indonesia
Imtitsal Ulya Salsabila School of Computing, Universitas Telkom, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.2.5158

Keywords:

Hyperparameter Tuning, Recommender System, SASRec, Self-Attention, Sequential Recommendation

Abstract

Sequential recommendation is a branch of Recommender Systems that aims to predict the next item a user will interact with based on their historical sequence of interactions. The main challenge in SR is to capture both short-term and long-term dependencies among items within a sequence. Self-Attentive Sequential Recommendation (SASRec) is a self-attention-based deep learning model designed to recognize sequential interaction patterns. Despite its effectiveness, the performance of SASRec is highly dependent on hyperparameter configurations, yet comprehensive evaluations remain limited. This research aims to evaluate the influence of SASRec's configuration through hyperparameter tuning on sequential recommendation performance. The hyperparameters used are hidden_size, inner_size, number of attention heads (num_heads), and number of layers (num_layers). The evaluation was conducted on two public datasets with different sparsity characteristics: MovieLens-1M (Sparsity ≈ 95.80%) and Amazon Musical Instruments (Sparsity ≈ 99.99%). In this study, Recall@k and MRR@k were used as performance metrics. The test results showed that hidden_size and inner_size had a significant positive impact on performance, especially on the dense dataset. The optimal hidden_size was obtained at hidden_size = 64 on the Amazon Musical Instrument dataset, and at hidden_size = 256 on the Movielens 1M dataset. The optimal inner_size was obtained at inner_size = 256 on both datasets. Meanwhile, the num_heads and num_layers hyperparameters did not provide a significant performance improvement. Furthermore, in the comparison between SASRec, GRU4Rec, and BERT4Rec, SASRec outperforms GRU4Rec and BERT4Rec in handling highly sparse datasets such as Amazon Musical Instruments obtained average recall@20 = 0.0678, and average MRR@20 = 0.0223.

Downloads

Download data is not yet available.

References

X. Bai, H. Peng, Y. Huang, J. Wang, Q. Yang, and A. Ramírez-De-Arellano, “A graph attention network integrated with gated spiking neural P systems for session-based recommendation,” Expert Syst Appl, vol. 286, Aug. 2025, doi: 10.1016/j.eswa.2025.128029.

D. B. Rajesh and A. Kumar, “Collaborative filtering models an experimental and detailed comparative study,” Sci Rep, vol. 15, no. 1, p. 31667, Aug. 2025, doi: 10.1038/s41598-025-15096-4.

H. Vaghari, M. Hosseinzadeh Aghdam, and H. Emami, “Group attention for collaborative filtering with sequential feedback and context aware attributes,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-94256-y.

M. B. S. Siddik and A. T. Wibowo, “Collaborative Filtering Based Food Recommendation System Using Matrix Factorization,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 7, no. 3, p. 1041, Jul. 2023, doi: 10.30865/mib.v7i3.6049.

H. U. Khan, A. Naz, F. K. Alarfaj, and N. Almusallam, “A transformer-based architecture for collaborative filtering modeling in personalized recommender systems,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-08931-1.

G. D. S. P. Moreira, S. Rabhi, J. M. Lee, R. Ak, and E. Oldridge, “Transformers4Rec: Bridging the Gap between NLP and sequential/session-based recommendation,” in RecSys 2021 - 15th ACM Conference on Recommender Systems, Association for Computing Machinery, Inc, Sep. 2021, pp. 143–153. doi: 10.1145/3460231.3474255.

H. Fan et al., “TiM4Rec: An efficient sequential recommendation model based on time-aware structured state space duality model,” Neurocomputing, vol. 654, p. 131270, Nov. 2025, doi: 10.1016/j.neucom.2025.131270.

Z. Wang, B. Liu, W. Huang, T. Hao, H. Zhou, and Y. Guo, “Leveraging multimodal large language model for multimodal sequential recommendation,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-14251-1.

Q. Tan et al., “Sparse-Interest Network for Sequential Recommendation,” in WSDM 2021 - Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Association for Computing Machinery, Inc, Aug. 2021, pp. 598–606. doi: 10.1145/3437963.3441811.

Z. Fan et al., “Sequential Recommendation via Stochastic Self-Attention,” in WWW 2022 - Proceedings of the ACM Web Conference 2022, Association for Computing Machinery, Inc, Apr. 2022, pp. 2036–2047. doi: 10.1145/3485447.3512077.

H. Wang, Y. Fan, Y. Du, X. Li, and X. Wang, “Improving contrastive learning with explanation method for sequential recommendation,” Expert Syst Appl, vol. 291, Oct. 2025, doi: 10.1016/j.eswa.2025.128534.

X. Du et al., “Frequency Enhanced Hybrid Attention Network for Sequential Recommendation,” in SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery, Inc, Jul. 2023, pp. 78–88. doi: 10.1145/3539618.3591689.

A. Klenitskiy and A. Vasilev, “Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?,” in Proceedings of the 17th ACM Conference on Recommender Systems, New York, NY, USA: ACM, Sep. 2023, pp. 1120–1125. doi: 10.1145/3604915.3610644.

D. Tikhonovich et al., “eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashion,” in Proceedings of the Nineteenth ACM Conference on Recommender Systems, New York, NY, USA: ACM, Sep. 2025, pp. 1175–1180. doi: 10.1145/3705328.3759317.

A. Volodkevich, D. Gusak, A. Klenitskiy, A. Pembek, and A. Vasilev, “Autoregressive generation strategies for Top-K sequential recommendations,” User Model User-adapt Interact, vol. 35, no. 3, p. 13, Sep. 2025, doi: 10.1007/s11257-025-09433-5.

U. Javed, K. Shaukat, I. A. Hameed, F. Iqbal, T. M. Alam, and S. Luo, “A Review of Content-Based and Context-Based Recommendation Systems,” International Journal of Emerging Technologies in Learning, vol. 16, no. 3, pp. 274–306, 2021, doi: 10.3991/ijet.v16i03.18851.

L. Pan, W. Pan, M. Wei, H. Yin, and Z. Ming, “A Survey on Sequential Recommendation,” Front Comput Sci, pp. 1–45, Mar. 2025, doi: 10.1007/s11704-025-41329-w.

W. Li et al., “Collaborative local–global context modeling for session-based recommendation,” Inf Process Manag, vol. 62, no. 5, p. 104196, Sep. 2025, doi: 10.1016/j.ipm.2025.104196.

A. H. J. P. Juni Permana and Agung Toto Wibowo, “Movie Recommendation System Based on Synopsis Using Content-Based Filtering with TF-IDF and Cosine Similarity,” International Journal on Information and Communication Technology (IJoICT), vol. 9, no. 2, pp. 1–14, Dec. 2023, doi: 10.21108/ijoict.v9i2.747.

W. Gao, “Research on optimization of library book recommendation system based on the collaborative fusion of transformer architecture and adaptive extreme learning machine,” Systems and Soft Computing, vol. 7, Dec. 2025, doi: 10.1016/j.sasc.2025.200287.

W. Zhang, Z. Chen, H. Zha, and J. Wang, “Learning from Substitutable and Complementary Relations for Graph-based Sequential Product Recommendation,” ACM Trans Inf Syst, vol. 40, no. 2, Apr. 2022, doi: 10.1145/3464302.

X. Huang, J. Sang, J. Yu, and C. Xu, “Learning to Learn a Cold-start Sequential Recommender,” ACM Trans Inf Syst, vol. 40, no. 2, pp. 1–25, Apr. 2022, doi: 10.1145/3466753.

Y. Ding, Y. Ma, W. K. Wong, and T. S. Chua, “Leveraging two types of global graph for sequential fashion recommendation,” in ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval, Association for Computing Machinery, Inc, Aug. 2021, pp. 73–81. doi: 10.1145/3460426.3463638.

X. Chen and Q. Li, “Causality-driven User Modeling for Sequential Recommendations over Time,” in WWW 2024 Companion - Companion Proceedings of the ACM Web Conference, Association for Computing Machinery, Inc, May 2024, pp. 1400–1406. doi: 10.1145/3589335.3651896.

M. Ma et al., “Improving Transformer-based Sequential Recommenders through Preference Editing,” ACM Trans Inf Syst, vol. 41, no. 3, pp. 1–24, Jul. 2023, doi: 10.1145/3564282.

W.-C. Kang and J. McAuley, “Self-Attentive Sequential Recommendation,” in 2018 IEEE International Conference on Data Mining (ICDM), IEEE, Nov. 2018, pp. 197–206. doi: 10.1109/ICDM.2018.00035.

K. Zhou, H. Yu, W. X. Zhao, and J. R. Wen, “Filter-enhanced MLP is All You Need for Sequential Recommendation,” in WWW 2022 - Proceedings of the ACM Web Conference 2022, Association for Computing Machinery, Inc, Apr. 2022, pp. 2388–2399. doi: 10.1145/3485447.3512111.

J. Ni, J. Li, and J. McAuley, “Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Stroudsburg, PA, USA: Association for Computational Linguistics, 2019, pp. 188–197. doi: 10.18653/v1/D19-1018.

F. M. Harper and J. A. Konstan, “The MovieLens Datasets,” ACM Trans Interact Intell Syst, vol. 5, no. 4, pp. 1–19, Jan. 2016, doi: 10.1145/2827872.

F. Sun et al., “Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer,” in International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, Nov. 2019, pp. 1441–1450. doi: 10.1145/3357384.3357895.

C. Chen, F. Liu, Y. Li, C. Wu, T. Qi, and Q. Liu, “User Behavior Modeling with Self-Supervised Contrastive Learning for Recommendation,” in Proceedings of the Web Conference 2021 (WWW), 2021.