Corpus Analysis and Annotation for Helpful Sentences in Product Reviews

  •  Hana Almagrabi    
  •  Areej Malibari    
  •  John McNaught    


For the last two decades, various studies on determining the quality of online product reviews have been concerned with the classification of complete documents into helpful or unhelpful classes using supervised learning methods. As in any supervised machine-learning task, a manually annotated corpus is required to train a model. Corpora annotated for helpful product reviews are an important resource for the understanding of what makes online product reviews helpful and of how to rank them according to their quality. However, most corpora for helpfulness are annotated on the document level: the full review. Little attention has been paid to carrying out a deeper analysis of helpful comments in reviews. In this article, a new annotation scheme is proposed to identify helpful sentences from each product review in the dataset. The annotation scheme, guidelines and the inter-annotator agreement scores are presented and discussed. A high level of inter-annotator agreement is obtained, indicating that the annotated corpus is suitable to support subsequent research.

This work is licensed under a Creative Commons Attribution 4.0 License.
  • ISSN(Print): 1913-8989
  • ISSN(Online): 1913-8997
  • Started: 2008
  • Frequency: quarterly

Journal Metrics

WJCI (2021): 0.557

Impact Factor 2021 (by WJCI):  0.304

h-index (December 2022): 40

i10-index (December 2022): 179

h5-index (December 2022): N/A

h5-median(December 2022): N/A

( The data was calculated based on Google Scholar Citations. Click Here to Learn More. )