DOI : https://doi.org/10.18517/ijaseit.14.5.20063

Fine-tuning GAN Models with Unpaired Aerial Images for RGB to NDVI Translation in Vegetation Index Estimation

Hendri Darmawan (1), Mike Yuliana (2), Moch. Zen Samsono Hadi (3), Rahardhita Widyatra Sudibyo (4)
(1) Department of Informatics and Computer Engineering, Politeknik Elektronika Negeri Surabaya (PENS), Surabaya, Indonesia
(2) Department of Informatics and Computer Engineering, Politeknik Elektronika Negeri Surabaya (PENS), Surabaya, Indonesia
(3) Department of Informatics and Computer Engineering, Politeknik Elektronika Negeri Surabaya (PENS), Surabaya, Indonesia
(4) Department of Informatics and Computer Engineering, Politeknik Elektronika Negeri Surabaya (PENS), Surabaya, Indonesia
Fulltext View | Download
How to cite (IJASEIT) :
Darmawan , Hendri, et al. “Fine-Tuning GAN Models With Unpaired Aerial Images for RGB to NDVI Translation in Vegetation Index Estimation”. International Journal on Advanced Science, Engineering and Information Technology, vol. 14, no. 5, Oct. 2024, pp. 1654-62, doi:10.18517/ijaseit.14.5.20063.
Citation Format :
Calculating the Normalized Difference Vegetation Index (NDVI) requires expensive multispectral cameras, posing challenges such as high costs and the need for technical expertise. This research aims to develop a method to transform RGB images obtained from Unmanned Aerial Vehicle (UAV) into NDVI images. Our proposed method leverages the CycleGAN model, an unsupervised image-to-image translation framework, to learn the intricate mapping between RGB values and NDVI. The model was trained on unpaired datasets of RGB and NDVI images, sourced from paddy fields located in Gresik and Yogyakarta, Indonesia. This process successfully encapsulated the complex correlation between these two modalities. Various training strategies were systematically investigated, including weight initialization schemes, fine-tuning procedures, and learning rate policies, to optimize the model's performance. The fine-tuned CycleGAN demonstrated superior performance in creating synthetic NDVI images from unpaired dataset, surpassing other methods in terms of fidelity, quality, and structural coherence. The results were impressive, with a Normalized Root Mean Square Error (NRMSE) of 0.327, a Peak Signal-to-Noise Ratio (PSNR) of 16.330, an Oriented FAST and Rotated BRIEF (ORB) score of 0.859, and a Structural Similarity Index (SSIM) of 0.757. The best performing CycleGAN model was then deployed on a low-spec microcomputer device, specifically the Raspberry Pi 4B with an average computation time of 21.0077 seconds. Raspberry Pi 4B was chosen for its lightweight, compact dimensions, and compatibility with efficient battery power connections, making it suitable for drone deployment.

J. J. Chen, S. Zhen, and Y. Sun, “Estimating Leaf Chlorophyll Content of Buffaloberry Using Normalized Difference Vegetation Index Sensors,” HortTechnology, vol. 31, no. 3, pp. 297–303, Jun. 2021, doi:10.21273/horttech04808-21.

M. Y. Abu Sari, Y. M. Mohmad Hassim, R. Hidayat, and A. Ahmad, “Monitoring Rice Crop and Paddy Field Condition Using UAV RGB Imagery,” JOIV : International Journal on Informatics Visualization, vol. 5, no. 4, p. 469, Dec. 2021, doi: 10.30630/joiv.5.4.742.

F. Kazemi and E. Ghanbari Parmehr, “Evaluation of RGB Vegetation Indices Derived from UAV Images for Rice Crop Growth Monitoring,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. X-4/W1-2022, pp. 385–390, Jan. 2023, doi:10.5194/isprs-annals-x-4-w1-2022-385-2023.

K. Upendar, K. N. Agrawal, N. S. Chandel, and K. Singh, “Greenness identification using visible spectral colour indices for site specific weed management,” Plant Physiology Reports, vol. 26, no. 1, pp. 179–187, Jan. 2021, doi: 10.1007/s40502-020-00562-0.

A. L. A. Ribeiro et al., “Vegetation Indices for Predicting the Growth and Harvest Rate of Lettuce,” Agriculture, vol. 13, no. 5, p. 1091, May 2023, doi: 10.3390/agriculture13051091.

L. S. Eng, R. Ismail, W. Hashim, and A. Baharum, “The Use of VARI, GLI, and VIgreen Formulas in Detecting Vegetation In aerial Images,” International Journal of Technology, vol. 10, no. 7, p. 1385, Nov. 2019, doi: 10.14716/ijtech.v10i7.3275.

J. Jiang et al., “Using Digital Cameras on an Unmanned Aerial Vehicle to Derive Optimum Color Vegetation Indices for Leaf Nitrogen Concentration Monitoring in Winter Wheat,” Remote Sensing, vol. 11, no. 22, p. 2667, Nov. 2019, doi: 10.3390/rs11222667.

E. Estrada-Peraza, E. Alvarez-Huezo, G. Girón-Morales, and Y. Rodriguez-Gallo, “RGB Image-Based Coffee Rust Detection: Application of Vegetation Indices and Algorithm Development,” 2023 IEEE Central America and Panama Student Conference (CONESCAPAN), vol. 13, pp. 23–28, Sep. 2023, doi: 10.1109/conescapan60431.2023.10328429.

H. Yang, J. Liu, H. Sun, and H. Zhang, “PACL: Piecewise Arc Cotangent Decay Learning Rate for Deep Neural Network Training,” IEEE Access, vol. 8, pp. 112805–112813, 2020, doi: 10.1109/access.2020.3002884.

J. Raitoharju, “Convolutional neural networks,” Deep Learning for Robot Perception and Cognition, pp. 35–69, 2022, doi: 10.1016/b978-0-32-385787-1.00008-7.

H. Darmawan, M. Yuliana, and Moch. Z. Samsono Hadi, “GRU and XGBoost Performance with Hyperparameter Tuning Using GridSearchCV and Bayesian Optimization on an IoT-Based Weather Prediction System,” International Journal on Advanced Science, Engineering and Information Technology, vol. 13, no. 3, pp. 851–862, Jun. 2023, doi: 10.18517/ijaseit.13.3.18377.

C. Davidson, V. Jaganathan, A. N. Sivakumar, J. M. P. Czarnecki, and G. Chowdhary, “NDVI/NDRE prediction from standard RGB aerial imagery using deep learning,” Computers and Electronics in Agriculture, vol. 203, p. 107396, Dec. 2022, doi: 10.1016/j.compag.2022.107396.

H. Darmawan, M. Yuliana, and Moch. Z. S. Hadi, “Cloud-based Paddy Plant Pest and Disease Identification using Enhanced Deep Metric Learning and k-NN Classification with Augmented Latent Fusion,” International Journal of Intelligent Engineering and Systems, vol. 16, no. 6, pp. 158–170, Dec. 2023, doi: 10.22266/ijies2023.1231.14.

J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks,” 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251, Oct. 2017, doi: 10.1109/iccv.2017.244.

I. Goodfellow et al., “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, Oct. 2020, doi: 10.1145/3422622.

P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976, Jul. 2017, doi: 10.1109/cvpr.2017.632.

M. Wojciuk, Z. Swiderska-Chadaj, K. Siwek, and A. Gertych, “Improving classification accuracy of fine-tuned CNN models: Impact of hyperparameter optimization,” Heliyon, vol. 10, no. 5, p. e26586, Mar. 2024, doi: 10.1016/j.heliyon.2024.e26586.

B. Yan, Z. Yang, H. Sun, and C. Wang, “ADE-CycleGAN: A Detail Enhanced Image Dehazing CycleGAN Network,” Sensors, vol. 23, no. 6, p. 3294, Mar. 2023, doi: 10.3390/s23063294.

X. Xu et al., “Artificial intelligence driven digital whole slide image for intelligent recognition of different development stages of tongue tumor via a new deep learning framework,” Engineering Reports, vol. 6, no. 1, Jun. 2023, doi: 10.1002/eng2.12706.

H. Darmawan, M. Yuliana, and Moch. Z. Samsono Hadi, “Realtime Weather Prediction System Using GRU with Daily Surface Observation Data from IoT Sensors,” 2022 International Electronics Symposium (IES), vol. vii, pp. 221–226, Aug. 2022, doi: 10.1109/ies55876.2022.9888468.

T. Banet, A. G. Smith, R. McGrail, D. H. McNear, and H. Poffenbarger, “Toward improved image‐based root phenotyping: Handling temporal and cross‐site domain shifts in crop root segmentation models,” The Plant Phenome Journal, vol. 7, no. 1, Jan. 2024, doi: 10.1002/ppj2.20094.

Y. Chen, A. Janowczyk, and A. Madabhushi, “Quantitative Assessment of the Effects of Compression on Deep Learning in Digital Pathology Image Analysis,” JCO Clinical Cancer Informatics, no. 4, pp. 221–233, Nov. 2020, doi: 10.1200/cci.19.00068.

S. Rani, A. Kurniawardhani, and Y. A. W. Rendani, “Steganography on Digital Color Image Using Modulo Function and Pseudo-Random Number Generator,” International Journal on Advanced Science, Engineering and Information Technology, vol. 11, no. 6, p. 2470, Dec. 2021, doi: 10.18517/ijaseit.11.6.12687.

F. D. Adhinata, A. Harjoko, and - Wahyono, “Object Searching on Real-Time Video Using Oriented FAST and Rotated BRIEF Algorithm,” International Journal on Advanced Science, Engineering and Information Technology, vol. 11, no. 6, p. 2518, Dec. 2021, doi: 10.18517/ijaseit.11.6.12043.

K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Image Quality Assessment: Unifying Structure and Texture Similarity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2020, doi: 10.1109/tpami.2020.3045810.

I. H. Suvon et al., “Business Category Classification via Indistinctive Satellite Image Analysis Using Deep Learning,” International Journal on Advanced Science, Engineering and Information Technology, vol. 13, no. 6, pp. 2219–2230, Dec. 2023, doi: 10.18517/ijaseit.13.6.19059.

J. Peng et al., “Implementation of the structural SIMilarity (SSIM) index as a quantitative evaluation tool for dose distribution error detection,” Medical Physics, vol. 47, no. 4, pp. 1907–1919, Jan. 2020, doi: 10.1002/mp.14010.

H. Tarek, H. Aly, S. Eisa, and M. Abul-Soud, “Optimized Deep Learning Algorithms for Tomato Leaf Disease Detection with Hardware Deployment,” Electronics, vol. 11, no. 1, p. 140, Jan. 2022, doi: 10.3390/electronics11010140.

A. Von Eye, W. Wiedermann, and S. Von Weber, “Configural analysis of oscillating progression,” Journal for Person-Oriented Research, vol. 7, no. 1, pp. 14–21, Aug. 2021, doi: 10.17505/jpor.2021.23448.

E. Hattula, L. Zhu, J. Raninen, J. Oksanen, and J. Hyyppä, “Advantages of Using Transfer Learning Technology with a Quantative Measurement,” Remote Sensing, vol. 15, no. 17, p. 4278, Aug. 2023, doi: 10.3390/rs15174278.

Md. J. Uddin, Y. Li, Md. A. Sattar, Z. Most. Nasrin, and C. Lu, “Effects of Learning Rates and Optimization Algorithms on Forecasting Accuracy of Hourly Typhoon Rainfall: Experiments With Convolutional Neural Network,” Earth and Space Science, vol. 9, no. 3, Mar. 2022, doi: 10.1029/2021ea002168.

W. Nawaz, M. H. Siddiqi, and A. Almadhor, “Adaptively Directed Image Restoration Using Resilient Backpropagation Neural Network,” International Journal of Computational Intelligence Systems, vol. 16, no. 1, May 2023, doi: 10.1007/s44196-023-00259-w.

V. Mudeng, M. Kim, and S. Choe, “Prospects of Structural Similarity Index for Medical Image Analysis,” Applied Sciences, vol. 12, no. 8, p. 3754, Apr. 2022, doi: 10.3390/app12083754.

J. Zhang, “Research on the algorithm of image feature detection and matching,” Applied and Computational Engineering, vol. 5, no. 1, pp. 527–535, May 2023, doi: 10.54254/2755-2721/5/20230636.

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
    2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
    3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).