Feature Selection Using Genetic Algorithm Combined with Linear Discriminant Analysis and k-Nearest Neighbors for High-Dimensional Text Classification

Eko Puji Laksono; M Zainal  Arifin; Aji Prasetya Wibawa

doi:10.52436/1.jutif.2026.7.4.5423

Authors

Eko Puji Laksono Departemen Teknik Elektro dan Informatika, Universitas Negeri Malang, Indonesia
M Zainal Arifin Departemen Teknik Elektro dan Informatika, Universitas Negeri Malang, Indonesia
Aji Prasetya Wibawa Departemen Teknik Elektro dan Informatika, Universitas Negeri Malang, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.4.5423

Keywords:

Genetic Algorithm, Klasifikasi Teks, Seleksi Fitur, Linear Discriminant Analysis, k-Nearest Neighbors, Optimisasi Hibrida

Abstract

Text classification is an important task in natural language processing that faces the main challenge of high dimensionality in the representation of text features, such as Bag-of-Words and TF-IDF. High dimensionality leads to data sparsity, increased computational complexity, and decreased model performance due to irrelevant and redundant features. Therefore, an effective feature selection method is needed to improve the accuracy and efficiency of text classification. This study aims to develop an optimal feature selection method by integrating Genetic Algorithm (GA) as a wrapper method, Linear Discriminant Analysis (LDA) for dimension reduction, and k-Nearest Neighbors (k-NN) as a classifier in a high-dimensional text classification pipeline. GA is used to explore a subset of features in an adaptive and global manner, LDA transforms features into spaces with maximum class separability, and k-NN classifies based on proximity within optimized feature spaces. Experiments were conducted on three standard text datasets, namely 20 Newsgroups, Reuters-21578, and BBC News, with a comprehensive evaluation using a variety of preprocessing configurations and feature representations. The results showed that the GA+LDA+k-NN approach significantly improved the classification accuracy compared to traditional feature selection methods such as Chi-Square, Information Gain, and Mutual Information. This approach is also more robust against noise and is able to handle minority classes better. These findings confirm the urgency of using evolutionary optimization-based feature selection methods in high-dimensional text classification, which not only improves model performance but also provides a practical solution for large-scale and complex text data processing. Thus, this research makes an important contribution both theoretically and applicatively in the development of modern text classification systems.

Downloads

Download data is not yet available.

References

S. Mirjalili, M. Gandomi, and S. Z. Mirjalili, "Hybrid genetic algorithm and particle swarm optimization for optimizing high-dimensional functions," Applied Soft Computing, vol. 102, pp. 107081-107095, 2021.

B. Xue, M. Zhang, and W. N. Browne, "Particle swarm optimization for feature selection in classification: A multi-objective approach," IEEE Transactions on Cybernetics, vol. 53, no. 4, pp. 1927-1941, 2023.

X. Wang, L. Gao, and Y. Liu, "A hybrid algorithm combining genetic algorithm and gradient descent for neural network training," Neural Computing and Applications, vol. 34, no. 8, pp. 5823-5837, 2022.

J. Chen, L. Zhang, and Y. Wu, "Adaptive hybrid optimization algorithm for high-dimensional complex functions," Journal of Global Optimization, vol. 85, no. 2, pp. 351-375, 2023.

R. Poli, J. Kennedy, and T. Blackwell, "Particle swarm optimization: An overview," Swarm Intelligence, vol. 1, no. 1, pp. 33-57, 2021.

M. A. Bhatti, M. Ayoub, and S. Irfan, "Metaheuristic algorithms for optimization problems: A comprehensive review," Applied Sciences, vol. 12, no. 5, pp. 2759-2785, 2023.

Y. Liu, L. Gao, and M. Li, "Hybrid differential evolution algorithm with local search for constrained optimization," IEEE Transactions on Evolutionary Computation, vol. 27, no. 3, pp. 654-668, 2023.

A. Faris, I. Aljarah, and S. Mirjalili, "Grey wolf optimizer: Theory, literature review, and applications," Natural Computing, vol. 20, no. 2, pp. 335-371, 2021.

P. N. Suganthan, N. Hansen, and J. J. Liang, "Problem definitions and evaluation criteria for the CEC 2021 competition on real parameter optimization," Technical Report, 2021.

T. Bäck, D. B. Fogel, and Z. Michalewicz, Handbook of Evolutionary Computation, IOP Publishing, 2021.

S. Das and P. N. Suganthan, "Differential evolution: A survey of the state-of-the-art," IEEE Transactions on Evolutionary Computation, vol. 15, no. 1, pp. 4-31, 2022.

D. E. Goldberg and K. Deb, "A comparative analysis of selection schemes used in genetic algorithms," Foundations of Genetic Algorithms, vol. 1, pp. 69-93, 2021.

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer Series in Operations Research, 2023.

N. Ibrahim, L. Abualigah, and R. A. Zitar, "Hybrid gorilla troops optimizer with sine cosine algorithm for high-dimensional optimization problems," Neural Computing and Applications, vol. 35, no. 10, pp. 7325-7351, 2023.

C. A. C. Coello and N. C. Cortés, "Solving multiobjective optimization problems using an artificial immune system," Evolutionary Computation, vol. 14, no. 1, pp. 53-83, 2022.

A. E. Eiben and J. E. Smith, Introduction to Evolutionary Computing, 3rd ed., Springer, 2023.

L. N. de Castro and F. J. Von Zuben, "Learning and optimization using the immune system," IEEE Transactions on Evolutionary Computation, vol. 26, no. 5, pp. 1099-1115, 2022.

M. Dorigo, M. Birattari, and T. Stützle, "Ant colony optimization: Artificial ants as a computational intelligence technique," IEEE Computational Intelligence Magazine, vol. 1, no. 4, pp. 28-39, 2021.

R. Storn and K. Price, "Differential evolution: A simple and efficient heuristic for global optimization over continuous spaces," Journal of Global Optimization, vol. 41, no. 2, pp. 341-359, 2024.

Q. K. Al-Shammari, Z. A. Aghbari, and M. A. Al-Garadi, "Improved genetic algorithm for solving the traveling salesman problem," Computers & Industrial Engineering, vol. 159, pp. 107456-107472, 2021.

Ahamed, S. F., Karuppasamy, S., & Chinnaiyan, P. (2025). Clinical Text Classification for Tuberculosis Diagnosis Using Natural Language Processing and Deep Learning Model with Statistical Feature Selection Technique. Informatics, 12(3), 64. https://doi.org/10.3390/informatics12030064

{22] Ferdous, S. M., Newaz, S. N. E., Mugdha, S. B. S., & Uddin, M. (2024). Sentiment Analysis in the Transformative Era of Machine Learning: A Comprehensive Review. Statistics, Optimization & Information Computing, 13(1), 331–346. https://doi.org/10.19139/soic-2310-5070-2113

Jallipalli, S. B., & Duvvuru, R. (2025). Reviews based Patient-Centric Recommendation System: A Machine Learning Analysis for Smart Health Care Management. IEEE Access, 1–1. https://doi.org/10.1109/ACCESS.2025.3574503

Saeed, H. F., Alkhalisy, M. A. E., & Khudhur, K. M. (2025). Ensemble CNN with Feature Selection and Soft Voting for Document Classification. Ingénierie Des Systèmes d Information, 30(8), 1953–1962. https://doi.org/10.18280/isi.300802

A. Kathole, S. Patil, Dr. D. Jadhav, H. Pathak, and A. S. Mirge, “Development of student intent-based educational chatbot system with adaptive and attentive DTCN on symmetric convolution approach,” MethodsX, vol. 15, p. 103542, Dec. 2025, doi: 10.1016/j.mex.2025.103542.