Aspect-based sentiment analysis of religious tourism reviews using IndoBERT: a case study of sunan gunung jati tomb in cirebon

Abstract

Religious tourism destinations at Islamic pilgrimage sites in Indonesia attract millions of visitors annually, yet visitor experience at these sites is often affected by management challenges that have rarely been examined through computational methods. Conventional sentiment analysis, which assigns a single polarity to  an entire review, cannot capture the multi-faceted nature of visitor feedback where spiritual appreciation and experiential complaints coexist. This study applied Aspect-Based Sentiment Analysis (ABSA) to 682 Google Maps reviews of the Tomb of Sunan Gunung Jati in Cirebon, Indonesia. A domain-specific aspect taxonomy was developed through iterative calibration, resulting in four categories: Religious-Cultural Value, Facilities and Infrastructure, Management and Service, and Comfort and Disturbance. Inter-annotator agreement reached a Cohen's Kappa of 0.64, indicating substantial reliability. Three models were compared: Support Vector Machine with TF-IDF, IndoBERT, and IndoBERTweet. IndoBERT achieved the best macro-averaged F1-score of 0.727. The analysis identified contrasting sentiment patterns across aspects: the Religious-Cultural Value category was predominantly positive, while the Comfort and Disturbance category was predominantly negative, largely associated with complaints about aggressive begging and forced donations. Temporal analysis suggested a gradual decline in comfort-related complaints in recent reviews, though this observation is based on limited recent data and requires further verification. These findings offer preliminary insights that may support evidence-based management of pilgrimage sites, subject to validation with larger and more diverse datasets.