Journal of Computer Technology & Applications

Journal of Computer Technology & Applications Review Article

Evaluating the Efficiency of LLMs-SA (Sentiment Analysis) via Social Media Texts

  1. Nilesh Jain Department of Computer Science and Applications, Mandsaur University, Mandsaur

Abstract

Sentiment analysis (SA) is becoming popular in business and scientific communities as the processing of natural language (NLP), computational linguistics, text analytics, image-based processing or video- based processing is used in extracting and mining subjective information in the web, social network, etc. It is able to detect positive, negative or neutral information and can be selected to absorb polarity, sentiments, urgency and goals of mount importance. The majority of the world's data is unstructured, and this makes it challenging to determine the feelings of the people. This paper outlines a sentiment taxonomy based on a transformer-type large language model, RoBERTa, to classify textual data from tweets. The process includes data gathering, tokenization, feature extraction with the help of the Bag of Words approach, and classification with the RoBERTa model. The provided RoBERTa model has been experimentally evaluated using a Twitter dataset, and results show that it outperforms well- known classifiers such as Support Vector Classifier (SVC) and Naive Bayes (NB) with accuracies of 99.9% and F1-score of 99.89%. The outcomes support the usefulness of the model in contextual semantics on informal and noisy social media text. The proposed solution will provide an effective and scalable mechanism to analyze sentiments in real time in different areas, such as business intelligence, political analysis, and health monitoring.

Keywords

  • Deep Learning
  • Machine Learning
  • RoBERTa
  • large language models (LLMs)
  • natural language processing (NLP)
  • sentiment analysis
  • social media
  • text classification
  • twitter data

References (8)

  1. Drus Z, Khalid H. Sentiment analysis in social media and its application: Systematic literature review. Procedia computer science. 2019 Jan 1;161:707-14.
  2. McGarry K. Analyzing social media data using sentiment mining and bigram analysis for the recommendation of youtube videos. Information. 2023 Jul 16;14(7):408.
  3. Chaudhary JK, Tyagi S, Sharma HP, Akram SV, Sisodia DR, Kapila D. Machine learning model- based financial market sentiment prediction and application. In2023 3rd international conference on advance computing and innovative technologies in engineering (ICACITE) 2023 May 12 (pp. 1456-1459). IEEE.
  4. Ranjan M, Tiwari S, Sattar AM, Tatkar NS. A new approach for carrying out sentiment analysis of social media comments using natural language processing. Engineering Proceedings. 2024 Jan 17;59(1):181.
  5. RS NN, Priyadarshini J achine earning lgorithms for the diagnosis of lzheimer’s and Parkinson’s Disease International Journal Jul; ( )
  6. Rongala S, Pahune SA, Velu H, Mathur S. Leveraging Natural Language Processing and Machine Learning for Consumer Insights from Amazon Product Reviews. In2025 3rd International Conference on Smart Systems for applications in Electrical Sciences (ICSSES) 2025 Mar 21 (pp. 1-6). IEEE.
  7. Aryal SK, Prioleau H, Aryal S. Sentiment analysis across multiple african languages: A current benchmark. arXiv preprint arXiv:2310.14120. 2023 Oct 21.
  8. Pandya S. Comparative Analysis of Large Language Models and Traditional Methods for Sentiment Analysis of Tweets Dataset. Int. J. Innov. Sci. Res. Technol. 2024;9(12):1647-57.