Enhancing cyberbullying detection with RoBERTa: A transformer-based approach

This study investigates the effectiveness of transformer-based models, specifically RoBERTa, in detecting cyberbullying on social media platforms compared to traditional machine learning models such as Random Forest (RF) and Long Short-Term Memory (LSTM). Cyberbullying poses significant challenges due to the evolving nature of language and the anonymity provided by digital platforms. The research focuses on fine-tuning RoBERTa for cyberbullying detection and evaluates its performance using a comprehensive real world dataset comprising approximately 48,000 manually annotated tweets, categorised into various forms of cyberbullying, including explicit and subtle abuses related to ethnicity, age, gender, and other characteristics. Results show that RoBERTa achieved the highest accuracy of 83.9%, outperforming LSTM (77.7%) and RF (79.2%). It excelled in the religion category (accuracy: 96.06%, precision: 96.48%) and ethnicity (accuracy: 98.60%, precision: 98.28%). RF led in the age category (accuracy: 98.31%, precision: 94.54%), with RoBERTa closely behind at 97.53% accuracy. LSTM performed lower, especially in the gender category, with an accuracy of 88.45% and precision of 72.12%. Although the results highlight the effectiveness of RoBERTa in recognising subtle forms of cyberbullying, it faced challenges in real-time applications due to slower inference times and higher computational costs. This research highlights the importance of contextual understanding in cyberbullying detection and the potential of transformer-based models to improve accuracy.