Data Augmentation-Driven Predictive Performance Refinement in Multi-Model Convolutional Neural Network for Cocoa Ripeness Prediction

Apriani Apriani; I Nyoman  Switrayana; Rifqi Hammad; Pahrul Irfan; Gede Yogi  Pratama

doi:10.52436/1.jutif.2025.6.5.5298

Authors

Apriani Computer Science, Universitas Bumigora, Indonesia
I Nyoman Switrayana Computer Science, Universitas Bumigora, Indonesia
Rifqi Hammad Software Engineering, Universitas Bumigora, Indonesia
Pahrul Irfan Informatics Engineering, Universitas Mataram, Indonesia
Gede Yogi Pratama Computer Science, Universitas Bumigora, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2025.6.5.5298

Keywords:

Cocoa, Convolutional Neural Network, Data Augmentation, Prediction, Ripeness

Abstract

Timely and accurate prediction of cocoa fruit ripeness is critical for optimizing harvest schedules, improving yield quality, and supporting post-harvest processing. Conventional visual inspection methods are prone to subjectivity and inconsistencies, especially when distinguishing among multiple ripeness levels based on fruit age. This study proposes a deep learning approach that leverages multi-model convolutional neural network transfer learning combined with image data augmentation to classify cocoa fruit into four maturity stages derived from fruit age. An augmented dataset of cocoa fruit images was used to fine-tune five well-established pre-trained models: MobileNetV2, Xception, ResNet50, DenseNet121, and DenseNet169. Data augmentation techniques were employed to increase variability and improve model generalization. Model evaluation was conducted using a standard 80:20 training-to-testing split to ensure sufficient data for learning while preserving a representative test set across all ripeness classes. The results demonstrate that DenseNet169 consistently outperformed other models, achieving the highest average accuracy of 85,05%, followed by DenseNet121 84,06%. Across all models, the use of data augmentation led to notable performance gains, highlighting its importance in enhancing predictive capability and reducing overfitting. The proposed framework shows promising potential for automating ripeness classification in agricultural contexts, offering a robust, scalable, and accurate solution for intelligent cocoa harvest management. This work contributes to the growing application of deep learning in precision agriculture, particularly in addressing fine-grained classification problems using limited but enriched visual data.

Downloads

Download data is not yet available.

References

J. Neilson, A. Dwiartama, N. Fold, and D. Permadi, “Resource-based industrial policy in an era of global production networks: Strategic coupling in the Indonesian cocoa sector,” World Dev., vol. 135, no. 105045, pp. 1–12, 2020, doi: https://doi.org/10.1016/j.worlddev.2020.105045.

I. Idawati et al., “Cocoa farmers’ characteristics on climate variability and its effects on climate change adaptation strategy,” Glob. J. Environ. Sci. Manag., vol. 10, no. 1, 2024, doi: 10.22034/gjesm.2024.01.21.

E. Marsoro et al., Statistik Kakao Indonesia 2023, 8th ed. Jakarta: Badan Pusat Statistik Indonesia, 2024.

J. E. Kongor, M. Owusu, and C. Oduro-Yeboah, “Cocoa production in the 2020s: challenges and solutions,” CABI Agric. Bioscinece, vol. 5, no. 102, 2024, doi: https://doi.org/10.1186/s43170-024-00310-6.

A. M. Calvo, B. L. Botina, M. C. García, W. A. Cardona, A. C. Montenegro, and J. Criollo, “Dynamics of cocoa fermentation and its effect on quality,” Sci. Rep., vol. 11, no. 1, 2021, doi: 10.1038/s41598-021-95703-2.

A. P. Romero Vergel, A. V. Camargo Rodriguez, O. D. Ramirez, P. A. Arenas Velilla, and A. M. Gallego, “A Crop Modelling Strategy to Improve Cacao Quality and Productivity,” Plants, vol. 11, no. 2, 2022, doi: 10.3390/plants11020157.

R. Niikoi Kotey et al., “Effects of Fermentation Periods and Drying Methods on Postharvest Quality of Cocoa (Theobroma Cacao) Beans in Ghana,” J. Food Qual., vol. 2022, 2022, doi: 10.1155/2022/7871543.

N. N. Suh and E. L. Molua, “Cocoa production under climate variability and farm management challenges: Some farmers’ perspective,” J. Agric. Food Res., vol. 8, 2022, doi: 10.1016/j.jafr.2022.100282.

M. Santander et al., “Unravelling Cocoa Drying Technology: A Comprehensive Review of the Influence on Flavor Formation and QualityNo Title,” Foods, vol. 14, no. 5, 2025, doi: https://doi.org/10.3390/foods14050721.

E. Subroto, M. Djali, R. Indiarto, E. Lembong, and N. Baiti, “Microbiological Activity Affects Post-Harvest Quality of Cocoa (Theobroma cacao L.) Beans,” Horticulturae, vol. 9, no. 7, 2023, doi: https://doi.org/10.3390/horticulturae9070805.

L. Goya, J. E. Kongor, and S. de Pascual-Teresa, “From Cocoa to Chocolate: Effect of Processing on Flavanols and Methylxanthines and Their Mechanisms of Action,” Int. J. Mol. Sci., vol. 23, no. 22, 2022, doi: https://doi.org/10.3390/ijms232214365.

J. E. Kongor and D. Rahadian Aji Muhammad, “Processing of Cocoa and Development of Chocolate Beverages,” 2023.

R. Essah, D. Anand, and S. Singh, “An intelligent cocoa quality testing framework based on deep learning techniques,” Meas. Sensors, vol. 24, 2022, doi: 10.1016/j.measen.2022.100466.

K. J. Ayikpa, D. Mamadou, P. Gouton, and K. J. Adou, “Classification of Cocoa Pod Maturity Using Similarity Tools on an Image Database: Comparison of Feature Extractors and Color Spaces,” Data, vol. 8, no. 6, 2023, doi: 10.3390/data8060099.

J. Y. Goh, Y. M. Yunos, and M. S. M. Ali, “Fresh Fruit Bunch Ripeness Classification Methods: A Review,” Food Bioprocess Technol., vol. 18, pp. 183–206, 2024, doi: https://doi.org/10.1007/s11947-024-03483-0.

A. Julca-Otiniano et al., “New Races of Hemileia vastatrix Detected in Peruvian Coffee Fields,” Agronomy, vol. 14, no. 8, p. 1811, 2024, doi: https://doi.org/10.3390/agronomy14081811.

M. Rizzo, M. Marcuzzo, A. Zangari, A. Gasparetto, and A. Albarelli, “Fruit ripeness classification: A survey,” Artificial Intelligence in Agriculture, vol. 7. 2023, doi: 10.1016/j.aiia.2023.02.004.

W. Chen et al., “MLP-based multimodal tomato detection in complex scenarios: Insights from task-specific analysis of feature fusion architectures,” Agriculture, vol. 221, 2024, doi: https://doi.org/10.1016/j.compag.2024.108951.

J. Lu, L. Tan, and H. Jiang, “Review on convolutional neural network (CNN) applied to plant leaf disease classification,” Agriculture (Switzerland), vol. 11, no. 8. 2021, doi: 10.3390/agriculture11080707.

N. Mamat, M. F. Othman, R. Abdoulghafor, S. B. Belhaouari, N. Mamat, and S. F. Mohd Hussein, “Advanced Technology in Agriculture Industry by Implementing Image Annotation Technique and Deep Learning Approach: A Review,” Agriculture (Switzerland), vol. 12, no. 7. 2022, doi: 10.3390/agriculture12071033.

M. Altalak, M. A. Uddin, A. Alajmi, and A. Rizg, “Smart Agriculture Applications Using Deep Learning Technologies: A Survey,” Appl. Sci., vol. 12, no. 12, 2022, doi: 10.3390/app12125919.

V. G. Dhanya et al., “Deep learning based computer vision approaches for smart agricultural applications,” Artificial Intelligence in Agriculture, vol. 6. 2022, doi: 10.1016/j.aiia.2022.09.007.

A. Bouguettaya, H. Zarzour, A. Kechida, and A. M. Taberkit, “Deep learning techniques to classify agricultural crops through UAV imagery: a review,” Neural Computing and Applications, vol. 34, no. 12. 2022, doi: 10.1007/s00521-022-07104-9.

Y. Yuan, L. Chen, H. Wu, and L. Li, “Advanced agricultural disease image recognition technologies: A review,” Information Processing in Agriculture, vol. 9, no. 1. 2022, doi: 10.1016/j.inpa.2021.01.003.

A. Khan, A. D. Vibhute, S. Mali, and C. H. Patil, “A systematic review on hyperspectral imaging technology with a machine and deep learning methodology for agricultural applications,” Ecological Informatics, vol. 69. 2022, doi: 10.1016/j.ecoinf.2022.101678.

S. Geerthik, G. A. Senthil, K. J. Oliviya, and R. Keerthana, “A System and Method for Fruit Ripeness Prediction Using Transfer Learning and CNN,” 2024, doi: https://doi.org/10.1109/IC3IoT60841.2024.10550209.

N. Aherwadi, U. Mittal, J. Singla, N. Z. Jhanjhi, A. Yassine, and M. S. Hossain, “Prediction of Fruit Maturity, Quality, and Its Life Using Deep Learning Algorithms,” Electron., vol. 11, no. 24, 2022, doi: 10.3390/electronics11244100.

C. Prasad, S. Kumar, and S. D. Rathod, “Intelligent System for Predicting Orange Fruit Ripeness by Integrating Climatic and Imaging Data using CNN Model for High-Accuracy Classification,” 2024, doi: https://doi.org/10.1109/ICAIQSA64000.2024.10882459.

J. K. Basak et al., “Prediction of physicochemical properties of strawberry fruits using convolutional neural network-regression models,” 2025. doi: https://doi.org/10.1007/s13580-025-00717-8.

D. Minagawa and J. Kim, “Prediction of Harvest Time of Tomato Using Mask R-CNN,” AgriEngineering, vol. 4, no. 2, 2022, doi: 10.3390/agriengineering4020024.

A. G. Said and B. Joshi, “SmartRipen: LSTM-GRU feature selection& XGBoost-CNN for fruit ripeness detection,” Food Phys., vol. 2, 2025, doi: https://doi.org/10.1016/j.foodp.2025.100053.

S. Han, J. Liu, G. Zhou, Y. Jin, M. Zhang, and S. Xu, “InceptionV3-LSTM: A Deep Learning Net for the Intelligent Prediction of Rapeseed Harvest Time,” Agronomy, vol. 12, no. 12, 2022, doi: 10.3390/agronomy12123046.

Y. Yu, J. Huang, L. Wang, and S. Liang, “A 1D-inception-ResNet based global detection model for thin-skinned multifruit spectral quantitative analysis,” Food Control, vol. 167, no. 110823, 2025, doi: https://doi.org/10.1016/j.foodcont.2024.110823.

N. Begum and M. K. Hazarika, “Maturity detection of tomatoes using transfer learning,” Meas. Food, vol. 7, 2022, doi: 10.1016/j.meafoo.2022.100038.

Harsh Mundhada, Sanskriti Sood, Saitejaswi Sanagavarapu, Rina Damdoo, and Kanak Kalyani, “Fruit Detection and Three-Stage Maturity Grading Using CNN,” Int. J. Next-Generation Comput., 2023, doi: 10.47164/ijngc.v14i1.1099.

H. Sun, S. Zhang, R. Ren, and L. Su, “Maturity Classification of ‘Hupingzao’ Jujubes with an Imbalanced Dataset Based on Improved MobileNet V2,” Agric., vol. 12, no. 9, 2022, doi: 10.3390/agriculture12091305.

A. A. Abdelhamid, A. A. Alhussan, A.-S. T. Qenawy, A. M. Osman, A. M. Elshewey, and M. Eed, “Potato Harvesting Prediction Using an Improved ResNet-59 Model,” Potato Res, vol. 68, pp. 1049–1068, 2025, doi: https://doi.org/10.1007/s11540-024-09773-6.

Z. Wang et al., “An improved Faster R-CNN model for multi-object tomato maturity detection in complex scenarios,” Ecol. Inform., vol. 72, 2022, doi: 10.1016/j.ecoinf.2022.101886.

K. Sabanci, “Benchmarking of CNN Models and MobileNet-BiLSTM Approach to Classification of Tomato Seed Cultivars,” Sustain., vol. 15, no. 5, 2023, doi: 10.3390/su15054443.

E. Gautama, T. K. A. Rahman, and L. Kamelia, “Measurement Of Optimizer Performance On The EfficientNet Architecture In Convolutional Neural Network For Classification Of Matoa Maturity Levels,” 2024, doi: https://doi.org/10.1109/ICWT62080.2024.10674685.

P. Nahak, D. K. Pratihar, and A. K. DebView, “Tomato maturity stage prediction based on vision transformer and deep convolution neural networks,” Int. J. Hybrid Intell. Syst., vol. 21, no. 1, 2025, doi: https://doi.org/10.3233/HIS-240021.

C. Wang et al., “Application of Convolutional Neural Network-Based Detection Methods in Fresh Fruit Production: A Comprehensive Review,” Frontiers in Plant Science, vol. 13. 2022, doi: 10.3389/fpls.2022.868745.

F. Xiao, H. Wang, Y. Xu, and R. Zhang, “Fruit Detection and Recognition Based on Deep Learning for Automatic Harvesting: An Overview and Review,” Agronomy, vol. 13, no. 6. 2023, doi: 10.3390/agronomy13061625.

S. Espinoza, C. Aguilera, L. Rojas, and P. G. Campos, “Analysis of Fruit Images With Deep Learning: A Systematic Literature Review and Future Directions,” IEEE Access, vol. 12, pp. 3837–3859, 2023, doi: https://doi.org/10.1109/ACCESS.2023.3345789.

C. C. Ukwuoma, Q. Zhiguang, M. B. Bin Heyat, L. Ali, Z. Almaspoor, and H. N. Monday, “Recent Advancements in Fruit Detection and Classification Using Deep Learning Techniques,” Math. Probl. Eng., vol. 2022, 2022, doi: 10.1155/2022/9210947.

N. Ismail and O. A. Malik, “Real-time visual inspection system for grading fruits using computer vision and deep learning techniques,” Inf. Process. Agric., vol. 9, no. 1, 2022, doi: 10.1016/j.inpa.2021.01.005.

Y. Tang et al., “Optimization strategies of fruit detection to overcome the challenge of unstructured background in field orchard environment: a review,” Precision Agriculture, vol. 24, no. 4. 2023, doi: 10.1007/s11119-023-10009-9.

X. Liu, N. Li, Y. Huang, X. Lin, and Z. Ren, “A comprehensive review on acquisition of phenotypic information of Prunoideae fruits: Image technology,” Frontiers in Plant Science, vol. 13. 2023, doi: 10.3389/fpls.2022.1084847.

C. Neupane, M. Pereira, A. Koirala, and K. B. Walsh, “Fruit Sizing in Orchard: A Review from Caliper to Machine Vision with Deep Learning,” Sensors, vol. 23, no. 8. 2023, doi: 10.3390/s23083868.

H. Naito, K. Shimomoto, T. Fukatsu, F. Hosoi, and T. Ota, “Interoperability Analysis of Tomato Fruit Detection Models for Images Taken at Different Facilities, Cultivation Methods, and Times of the Day,” AgriEngineering, vol. 6, no. 2, pp. 1827–1846, 2024, doi: https://doi.org/10.3390/agriengineering6020106.

A. M. Hayajneh, S. Batayneh, E. Alzoubi, and M. Alwedyan, “TinyML Olive Fruit Variety Classification by Means of Convolutional Neural Networks on IoT Edge Devices,” AgriEngineering, vol. 5, no. 4, 2023, doi: 10.3390/agriengineering5040139.

I. N. Switrayana and M. Azwar, “Optimizing Scalability in Spice Identification through Transfer Learning with Convolutional Neural Networks,” vol. 11, no. 1, pp. 73–84, 2025, doi: 10.24014/coreit.v11i1.35453.

T. Kumar, R. Brennan, A. Mileo, and M. Bendechache, “Image Data Augmentation Approaches: A Comprehensive Survey and Future Directions,” IEEE Access, vol. 12, pp. 187536–187571, 2024, doi: 10.1109/ACCESS.2024.3470122.

M. Schwonberg, F. El Bouazati, N. M. Schmidt, and H. Gottschalk, “Augmentation-based Domain Generalization for Semantic Segmentation,” IEEE Intell. Veh. Symp. Proc., vol. 2023-June, 2023, doi: 10.1109/IV55152.2023.10186752.

T. B. Shahi, C. Sitaula, A. Neupane, and W. Guo, “Fruit classification using attention-based MobileNetV2 for industrial applications,” PLoS One, vol. 17, no. 2 February, pp. 1–21, 2022, doi: 10.1371/journal.pone.0264586.

F. Salim, F. Saeed, S. Basurra, S. N. Qasem, and T. Al-Hadhrami, “DenseNet-201 and Xception Pre-Trained Deep Learning Models for Fruit Recognition,” Electron., vol. 12, no. 14, 2023, doi: 10.3390/electronics12143132.

M. Han and C. Yi, “Deep Convolutional Neural Networks for Palm Fruit Maturity Classification,” pp. 1–9, 2025, [Online]. Available: http://arxiv.org/abs/2502.20223.

K. R. Ariawan, A. A. G. Ekayana, I. P. Y. Indrawan, K. R. Winatha, and I. N. A. F. Setiawan, “Performance Comparasion of DenseNet-121 and MobileNetV2 for Cacao Fruit Disease Image Classification,” Indones. J. Data Sci., vol. 6, no. 1, pp. 30–38, 2025, doi: 10.56705/ijodas.v6i1.233.

S. Nuanmeesri, “Enhanced hybrid attention deep learning for avocado ripeness classification on resource constrained devices,” Sci. Rep., vol. 15, no. 1, pp. 1–15, 2025, doi: 10.1038/s41598-025-87173-7.

I. N. Switrayana, S. Hadi, and N. Sulistianingsih, “A Robust Gender Recognition System using Convolutional Neural Network on Indonesian Speaker,” vol. 13, pp. 1008–1021, 2024.

J. F. Restrepo-Arias, M. I. Salinas-Agudelo, M. I. Hernandez-Pérez, A. Marulanda-Tobón, and M. C. Giraldo-Carvajal, “RipSetCocoaCNCH12: Labeled Dataset for Ripeness Stage Detection, Semantic and Instance Segmentation of Cocoa Pods,” Data, vol. 8, no. 6, 2023, doi: 10.3390/data8060112.

X. Pei et al., “Robustness of machine learning to color, size change, normalization, and image enhancement on micrograph datasets with large sample differences,” Mater. Des., vol. 232, p. 112086, 2023, doi: 10.1016/j.matdes.2023.112086.

I. N. Switrayana and N. U. Maulidevi, “Collaborative Convolutional Autoencoder for Scientific Article Recommendation,” Proc. - 2022 9th Int. Conf. Inf. Technol. Comput. Electr. Eng. ICITACEE 2022, pp. 96–101, 2022, doi: 10.1109/ICITACEE55701.2022.9924130.

K. Kayaalp, “A deep ensemble learning method for cherry classification,” Eur. Food Res. Technol., vol. 250, no. 5, pp. 1513–1528, 2024, doi: 10.1007/s00217-024-04490-3.

P. T. Huong, L. T. Hien, N. M. Son, H. C. Tuan, and T. Q. Nguyen, “Enhancing deep convolutional neural network models for orange quality classification using MobileNetV2 and data augmentation techniques,” J. Algorithms Comput. Technol., vol. 19, 2025, doi: 10.1177/17483026241309070.