Real-Time Extraction and Annotation of Social Media Contents for Predicting National Consumer Confidence Index

Abstract

The advent of web enabled technologies has given birth to new communication platforms such as Facebook, Twitter, YouTube, and blogsites. Online users belonging from variant geographical backgrounds share their opinion, sentiments, and appraisals about number of real-world entities on the social media platforms. These opinion bearing contents have great importance to observer and analysts. These opinionative contents can benefit in the prediction of consumer confidence index (CCI) which is referenced by businesses, governments, and other institutions when they make strategic decision. Social media channels and microblogging sites can have a high volume of data on consumer confidence, analyzing such contents can significantly improve the impact and accuracy of CCI but unavailability of consolidated application for the extraction of user generated content is restricting the further process. However, this study aims to conduct an implementation based comparative literature review to unfold the most valuable mechanisms of text extraction, normalization, and annotation of social media contents from Facebook, Twitter, Youtube, and blogging sites for effective prediction of the CCI. A case study of NACOP (Pakistani National Consumer Confidence Predictor) proposed by Ashraf et al. (2022) about the data of purchasing behavior, consumer price, Job/Employment and personal finance is presented to explore the data extraction tools for the Facebook, Twitter, Youtube, and blogging sites. The experimental evaluation revealed that Facepager, Tagv6, Netvizz, and web scrapper are the optimum extraction APIs for Facebook, Twitter, YouTube, and Blogsites respectively. The study has significant implications to theory and practice.