Understanding the Prediction Mechanism of Sentiments by XAI Visualization

Research output: Contribution to journalArticle

Abstract

People often rely on online reviews to make purchase decisions. The present work aimed to gain an understanding of a machine learning model's prediction mechanism by visualizing the effect of sentiments extracted from online hotel reviews with explainable AI (XAI) methodology. Study 1 used the extracted sentiments as features to predict the review ratings by five machine learning algorithms (knn, CART decision trees, support vector machines, random forests, gradient boosting machines) and identified random forests as best algorithm. Study 2 analyzed the random forests model by feature importance and revealed the sentiments joy, disgust, positive and negative as the most predictive features. Furthermore, the visualization of additive variable attributions and their prediction distribution showed correct prediction in direction and effect size for the 5-star rating but partially wrong direction and insufficient effect size for the 1-star rating. These prediction details were corroborated by a what-if analysis for the four top features. In conclusion, the prediction mechanism of a machine learning model can be uncovered by visualization of particular observations. Comparing instances of contrasting ground truth values can draw a differential picture of the prediction mechanism and inform decisions for model improvement.
Original languageEnglish
Journal International Conference on Natural Language Processing and Information Retrieval
Publication statusAccepted/In press - 2020 Mar 3

Bibliographical note

This is the author's prefinal version be published in conference proceedings: 4th International Conference on Natural Language Processing and Information Retrieval, Sejong, South Korea, 26-28 June, 2020, ACM

Fingerprint Dive into the research topics of 'Understanding the Prediction Mechanism of Sentiments by XAI Visualization'. Together they form a unique fingerprint.

  • Cite this