By Bert de Vink ·
When we built ReviewGraph, the question was simple: can a knowledge graph built from customer reviews predict the rating a guest gave? We tested it on 10,000 hotel reviews and compared it with classic text baselines and with GPT-4o. The honest answer is that it did about as well as the alternatives, and did not clearly beat them. That is exactly why the result is worth writing about.
What we built
Each review was turned into (subject, predicate, object) triples, such as a room and the way the guest described it, each with a sentiment score. Those triples became a graph in Neo4j. From the graph we took two kinds of features: Node2Vec embeddings that describe how a review sits in the network, and sentiment summaries per review: the average, the most negative and the most positive relationship.
Classifiers such as Random Forest, Logistic Regression and a small neural network then predicted the star rating. We compared this with Bag of Words, TF-IDF and Word2Vec baselines, and with GPT-4o predicting ratings from a sample of the same reviews.
What we found
- Between the best baseline, the best graph model and the LLM there was no substantial difference. The graph model reached a Cohen's Kappa similar to the LLM, with slightly lower accuracy.
- Small was better. The best configuration used only 5 embedding dimensions, and Random Forest worked best on these low-dimensional graph features but not on the baselines.
- Sentiment range mattered. Adding the minimum and maximum sentiment per review, not just the average, improved accuracy and especially Kappa. A review that mixes strong praise and strong criticism looks average if you only look at the mean.
- The graph structure alone already carried signal. Models using only the embeddings did surprisingly well.
Why a tie is still a result
If the only goal is the best possible number, a large language model is an easy choice, and this paper does not claim otherwise. But prediction accuracy is rarely the only thing a team needs. A graph gives you things a score does not.
- You can inspect it. Every prediction traces back to concrete nodes and relations, such as which aspects of a stay were mentioned and how positively.
- You can explore it. We built a visualisation where a user can filter reviews by score, highlight the most negative topics and open a single review to see what it is about.
- You can extend it. The same graph can feed retrieval-augmented generation, so an LLM summarises the graph rather than raw text, which is a direction we name as future work.
- The paper reports lower computational cost than the ensemble approaches it is compared with.
What we would do differently
The weakest part was our triple extraction. It was crude, produced many low-quality triples, and many nodes appeared only once and gave the model little to learn from. We think a fine-tuned language model for extraction would help most. Node2Vec also has to be retrained whenever a node is added, so graph neural networks such as GraphSAGE are the natural next step.
What it means for practice
This is our reading, not a claim from the paper. When data comes from many places and people need to understand and trust a prediction, the structure you build around the model can be worth more than a few points of accuracy. Start by making information connected and inspectable, then decide which model sits on top. That is how we approach knowledge graph projects.
The paper reports what worked and what did not, including the limits of our own extraction. Read it for the full method and numbers.
← All articles