Rethinking the role of three-star ratings: Handling inconsistency in indonesian tourism reviews usingcost-sensitive learning XGBoost

Abstract

Indonesian people often give contrary online reviews, for instance ratings that do not always match the actual feelings. This can make it difficult for tourism sector, such as Solo Safari, to handle complaints and improve service quality. Consequently, this study has formulated a rating prediction model using the XGBoost algorithm with a dataset of 2,047 reviews that have been relabeled. The best model, the FastText-XGBoost with Cost-Sensitive Learning, signify that it performs quite well with an average difference the prediction is only 0.04 points from the actual rating. However, the results are not optimal because the meaning of the review text still feels fuzzy even though being relabeled. This problem arises because reviewers tend to position sentiment very positively or negatively in the moderate category, so the perimeters between classes become less clear. This study then proposed an extreme class restructuring by simplifying the category by removing the 3-star rating. This method can increase the accuracy of the model to 0.9440 and clarify the category limits. Therefore, the model can be used on the service dashboard to help Solo Safari management respond to critical feedback faster.