Fine-Tuning Open-Source LLMs for Social Good: The Case of Cyber Falcon in Cyberbullying Detection

Abstract

Shopping is a very important issue of concern due to its ability to harm and affect the victims in a negative manner. In the recent past, there has been increasing literature that seeks to explore the application of machine learning models to identify acts of cyberbullying. The paper presents a new approach to detecting posts related to cyberbullying, which is based on the Large Language Model Falcon 7b. The training was performed on a set of 50,000 tweets where an effort was made to ensure that there was a balance of cyber- and non-cyberbullying was evenly distributed in the data set. Fine-tuning on the training dataset was then done on the model. After the fine-tuning, the analysis and evaluation of the specific dataset was fully carried out to establish the testing dataset. The above model returned a precision score of 0.334 and accuracy score of 0.334; both of which were much higher than the precision and accuracy scores of naive NLP detection model. In this research, it was found that Falcon 7b model may be a fresh approach to identify any cases of cyberbullying. It has a high capacity of learning complex patterns of language, and this factor is indicative of cyberbullying. Also, it shows the large competence in applying the gained knowledge in new environments, which depicts the ability of the generalization. Besides, performance by the model results in efficiency because it is able to classify a tweet within a few seconds time frame. The implications of the study findings on preventing and detecting cyberbullying in the future are greater. The discussed model has the potentials to be used in the creation of automated systems with a specific role of identifying the cases of cyberbullying. Such systems can be used on different social media. Such systems show that they can identify the cases of cyberbullying at the nascent stage and engage in measures to stop the damage that has been caused to people who victims of this practice. Besides, the adoption of this model demonstrates its ability to be further extended in its application to educate human moderators who are to monitor cases of cyberbullying. The ability of the moderators to identify and retrieve the cyberbullying messages in the social networking sites will be enhanced by attaining a balanced understanding of deliberate linguistic codes employed as recognizable indicators of cyberbullying. Thus, the research indicated that Falcon 7b model has a number of feasible elements that can be implemented in the sphere of cyberbullying detection. This model has great accuracy, efficiency and scalability, therefore a potentially marketable and innovative means of approaching and handling this monumental problem.