DOI : https://doi.org/10.18517/ijaseit.13.3.18273

Customer Satisfaction Assessment System on Transactions E-commerce Product Purchases Using Sentiment Analysis

Amil Ahmad Ilham (1), Anugrayani Bustamin (2), Eugenius Wahyudiarto (3)
(1) Informatics Department, Universitas Hasanuddin, Makassar, 90245, Indonesia
(2) Informatics Department, Universitas Hasanuddin, Makassar, 90245, Indonesia
(3) Informatics Department, Universitas Hasanuddin, Makassar, 90245, Indonesia
Fulltext View | Download
How to cite (IJASEIT) :
Ilham, Amil Ahmad, et al. “Customer Satisfaction Assessment System on Transactions E-Commerce Product Purchases Using Sentiment Analysis”. International Journal on Advanced Science, Engineering and Information Technology, vol. 13, no. 3, June 2023, pp. 1041-5, doi:10.18517/ijaseit.13.3.18273.
Citation Format :
Currently, more products appear, and various services that offer similar products make it difficult for buyers to decide to buy before seeing reviews from other users. The growth of different e-commerce platforms exacerbates this. Users spent more time choosing products on each platform with many alternative considerations, such as looking at ratings, prices, and reviews from other buyers. This study conducted the optimization process of selecting e-commerce products so that users do not have to spend a long time reading every review when they want to buy a product. This research is expected to provide a comprehensive assessment of the purchase transaction of a product from the reviews provided. The data is sourced from product reviews on e-commerce in Indonesia, which are then classified into positive, negative, and neutral sentiments. The data is divided into 10 folds of data using stratified k-fold cross-validation, consisting of training and testing data with ratios of 90% and 10% of the total data. Our research proposed a system that implemented our modified Naive Bayes model to calculate a product's Customer Satisfaction (CSAT) score and compare it with the Google Cloud NLP model. In our model, the log prior and log-likelihood formulas are modified in the algorithm, adding the prefix "NOT_" after the negation words in the preprocessing. This doubled our model’s F1 score and increased the accuracy by 32%, from 59% to 91%, when compared to the Naive Bayes algorithm without modification.

Z. Shahbazi and Y.-C. Byun, “Product Recommendation Based on Content-based Filtering Using XGBoost Classifier,” Int. J. Adv. Sci. Technol., vol. 29, no. 4, pp. 6979-6988, 2020.

G. Khanvilkar and Prof. Deepali Vora, “Sentiment Analysis for Product Recommendation Using Random Forest,” Int. J. Eng. Technol., vol. 7, no. 3.3, pp. 87-89, Jun. 2018, doi: 10.14419/ijet.v7i3.3.14492.

H. Parveen and S. Pandey, “Sentiment analysis on Twitter Data-set using Naive Bayes algorithm,” in 2016 2nd International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT), Bangalore, India, 2016, pp. 416-419. doi: 10.1109/ICATCCT.2016.7912034.

R. Alatrash, R. Priyadarshini, H. Ezaldeen, and A. Alhinnawi, “Augmented language model with deep learning adaptation on sentiment analysis for E-learning recommendation,” Cognitive System Research, vol. 75, pp. 53-69, Sep. 2022, doi.org/10.1016/j.cogsys.2022.07.002.

M. Bibi, W. A. Abbasi, A. Aziz, S. Khalil, M. Uddin, C.Iwendi, and T. R. Gadekallu, “A novel unsupervised ensemble framework using concept-based linguistic methods and machine learning for twitter sentiment analysis,” Pattern Recognition Letters, vol. 158, pp. 80-86, June 2022, doi.org/10.1016/j.patrec.2022.04.004

H. Deng, D. Ergu, F. Liu, Y. Cai and B. Ma, “Text sentiment analysis of fusion model based on attention mechanism,” Procedia Computer Science, vol 199, pp. 741-748, 2022, doi.org/10.1016/j.procs.2022.01.092

A.R. Rahmanti, C.Chien, A.A.Nursetyo, A. Husnayain, B. S. Wiratama, A. Fuad, H- Yang and Y.J. Li, “Social media sentiment analysis to monitor the performance of vaccination coverage during the early phase of the national COVID-19 vaccine rollout,” Computer Methods and Programs in Biomedicine, vol 221, June 2022, doi.org/10.1016/j.cmpb.2022.106838

I. N. K. Bayu, I. M. A. Suarjaya and P. W. Buana, "Classification of Indonesian Population's Level Happiness on Twitter Data Using N-Gram, Naí¯ve Bayes, and Big Data Technology," Int. J on Adv Scie Eng Inf Tech, vol. 12, no. 5, pp. 1944-1949, 2022

S. Farzadnia and I. R. Vanani, “Identification of opinion trends using sentiment analysis of airlines passengers' reviews,” Journal of Air Transport Management, vol. 103, August 2022, doi.org/10.1016/j.jairtraman.2022.102232

G. Hanane and N. El Habib, “Semisupervised neural biomedical sense disambiguation approach for aspect-based sentiment analysis on social networks,” Journal of Biomedical Inf, vol. 135, 2022, doi: 10.1016/j.jbi.2022.104229.

L. Chucu, F. Fan, L. Xu, C. Tie, T. Xu, L. Jianguo, and L. Xin, “Improving sentiment analysis accuracy with emoji embedding,” Journal of Safety Scie and Resilience, vol. 2, no. 4, pp. 246-252, 2021.

A. Shaha, A. Allulo, A. Asma, A. Amany, A. Sara, A. Nada & A. Aisha, "Customer Satisfaction Measurement using Sentiment Analysis." International Journal of Advanced Computer Science and Applications, vol. 9, no 10, 2018, doi:14569/IJACSA.2018.090216.

L. R. Shreyas, "Sentiment Analysis of Customer Satisfaction using Deep Learning," International Research Journal of Computer Science, vol.6, no.12, 2019.

A. Nair, C. Paralkar, J. Pandya, Y. Chopra, and D. Krishnan, “Comparative Review on Sentiment analysis-based Recommendation system,” in 6th Int. Conf. Converg. Technol. I2CT 2021, pp. 2-7, 2021, doi: 10.1109/I2CT51068.2021.9418222.

T. Hariguna and V. Rachmawati, “Community Opinion Sentiment Analysis on Social Media Using Naive Bayes Algorithm Methods,” IJIIS Int. J. Informatics Inf. Syst., vol. 2, no. 1, pp. 33-38, 2019, doi: 10.47738/ijiis.v2i1.11.

E. Asani, H. Vahdat-Nejad, and J. Sadri, “Restaurant recommender system based on sentiment analysis,” Mach. Learn. with Appl., vol. 6, no. July, p. 100114, 2021, doi: 10.1016/j.mlwa.2021.100114.

M. D. Prasetio, R. Y. Xavier, H. Rachmat, W. Wiyono, and D. S. E. Atmaja, “Sentiment analysis on myindihome user reviews using support vector machine and naí¯ve bayes classifier method,” Int. J. Ind. Optim., vol. 2, no. 2, p. 141, 2021, doi: 10.12928/ijio.v2i2.4449.

M. Ahmad, M. Octaviansyah, A. Kardiana, and K. Prasetyo, "Sentiment Analysis System of Indonesian Tweets using Lexicon and Naí¯ve Bayes Approach." in Fifth International Conf. on Informatics and Computing, pp.1-4, 2020.

J. Kim, D. Hwang, and H. Jung, “Product recommendation system based user purchase criteria and product reviews,” Int. J. Electr. Comput. Eng. IJECE, vol. 9, no. 6, p. 5454, Dec. 2019, doi: 10.11591/ijece.v9i6.pp5454-5462.

A. R. Hanni, M. M. Patil, and P. M. Patil, “Summarization of customer reviews for a product on a website using natural language processing,” in 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Jaipur, India, Sep. 2016, pp. 2280-2285. doi: 10.1109/ICACCI.2016.7732392.

P. Thota and E. Ramez, “Web Scraping of COVID-19 News Stories to Create Datasets for Sentiment and Emotion Analysis,” in the 14th PErvasive Technologies Related to Assistive Environments Conference, New York, NY, USA, Jun. 2021, pp. 306-314. doi: 10.1145/3453892.3461333.

L. Irshad, L. Yan, and Z. Ma, “Schema-Based JSON Data Stores in Relational Databases,” J. Database Manag. JDM, vol. 30, no. 3, pp. 38-70, Jul. 2019, doi: 10.4018/JDM.2019070103.

H.-T. Duong and T.-A. Nguyen-Thi, “A review: preprocessing techniques and data augmentation for sentiment analysis,” Comput. Soc. Netw., vol. 8, no. 1, p. 1, Jan. 2021, doi: 10.1186/s40649-020-00080-x.

F. Alzazah, X. Cheng, and X. Gao, “Predict Market Movements Based on the Sentiment of Financial Video News Sites,” in 2022 IEEE 16th International Conference on Semantic Computing (ICSC), Jan. 2022, pp. 103-110. doi: 10.1109/ICSC52841.2022.00022.

M. Wongkar and A. Angdresey, “Sentiment Analysis Using Naive Bayes Algorithm Of The Data Crawler: Twitter,” in 2019 Fourth International Conference on Informatics and Computing (ICIC), Oct. 2019, pp. 1-5. doi: 10.1109/ICIC47613.2019.8985884.

N. H. Khun and H. A. Thant, “Visualization of Twitter Sentiment during the Period of US Banned Huawei,” in 2019 International Conference on Advanced Information Technologies (ICAIT), Nov. 2019, pp. 274-279. doi: 10.1109/AITC.2019.8921014.

Z. Yang, S. Vijlbrief, and N. Okazaki, “TokyoTech_NLP at SemEval-2019 Task 3: Emotion-related Symbols in Emotion Detection,” in Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, Minnesota, USA, Jun. 2019, pp. 350-354. doi: 10.18653/v1/S19-2061.

H. M. Keerthi Kumar and B. S. Harish, “Classification of Short Text Using Various Preprocessing Techniques: An Empirical Evaluation,” in Recent Findings in Intelligent Computing Techniques, Singapore, 2018, pp. 19-30. doi: 10.1007/978-981-10-8633-5_3.

H.-H. CHEN, “The Contextual Analysis of Chinese Sentences with Punctuation Marks,” Lit. Linguist. Comput., vol. 9, no. 4, pp. 281-289, Jan. 1994, doi: 10.1093/llc/9.4.281.

M. Nabil, M. Aly, and A. Atiya, “LABR: A Large-Scale Arabic Sentiment Analysis Benchmark,” Computing and Language, May 2015. doi: 10.48550/arXiv.1411.6718.

D. Jurafsky and J. H. Martin, Speech and Language Processing, 3d Edition draft. web.stanford.edu, 2020. Accessed: Aug. 02, 2021. [Online]. Available: https://web.stanford.edu/~jurafsky/slp3/

G. Forman and M. Scholz, “Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement,” ACM SIGKDD Explor. Newsl., vol. 12, no. 1, pp. 49-57, Nov. 2010, doi: 10.1145/1882471.1882479.

Y. Huang and L. Li, “Naive Bayes classification algorithm based on small sample set,” in 2011 IEEE International Conference on Cloud Computing and Intelligence Systems, Sep. 2011, pp. 34-39. doi: 10.1109/CCIS.2011.6045027.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
    2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
    3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).