contentsift

Inspiration

Digital communication through social media has transcended all barriers. While X, a social media app, is great for consuming content of today’s biggest news, and events, following your favourite celebrities or simply communicating with your friends, it can also expose its users to hate speech, misinformation, and harmful content. Our team was inspired by the potential to create a safer digital space for X users. We aimed to make a tool that identifies such detrimental content and allows X users to sift through social media responsibly. Our web extension, ContentSift, is a manifestation of this vision, leveraging ML and NLP models to bring a layer of safety and reliability to everyday online interactions.

What it does

ContentSift is a Chrome Web Extension that allows you to moderate Twitter’s content feed so that the user can block out content that they don’t want to see, such as hate speech, violence and graphic content, sexually explicit content, as well as misinformation. It does this by using the Azure AI Content Safety API as well as the ClaimBuster API. The Azure AI Content Safety API flags for Content that contains hate speech, violence, self-harm, as well as sexual content, and the ClaimBuster API checks for misinformation.

How we built it

Our Chrome extension involved the development of two major parts, the Chrome extension and a backend API. Chrome Extension :

Functionality: Our extension actively scans content displayed in the X Chrome Browser. It identifies textual elements on web pages, parsing and delimiting them in preparation for analysis.
Technology: We leveraged Node.js for its efficiency in handling asynchronous events and its compatibility with Chrome’s extension development framework.

Backend API:

Workflow: A Flask application was developed to act as a server that interacts with both Chrome extension and two AI services: Azure AI content safety API and ClaimBusters API. When the front-end sends content data as an array of X posts, our backend first collates this information, batching up to 10 text items per request to optimize processing.
Content Analysis: Azure AI-Content Safety API was used for its robust capabilities in analyzing and categorizing content based on safety and appropriateness. This API helped us identify harmful content in four categories: Hate speech, self-harm, sexual content and violent content. For each of these categories, our API also shows the severity of each.
Misinformation Detection: We integrated the ClaimBusters API to tackle the challenge of misinformation. This powerful tool analyzes the text for factual accuracy and flags potential misinformation.
Efficient Processing: To reduce the load on both our server and API, and provide a seamless experience for the user, we analyze a batch of texts for both misinformation and content safety and send each batch of text to the front-end, which is the chrome extension.

Challenges we ran into

Some challenges that we ran into were finding an appropriate AI model that is able to detect misinformation and false statements, however, we decided to use Claim Buster’s API, which uses the google fact checker API to check whether the content contains false information. Although, the Google Fact Checker API was more complicated than we thought. This is due to the fact that Google Fact Checker API gives search results from multiple sources where some sources have different conclusions to the statement given. Additionally, working with Manifest v3 to produce a Chrome Extension proved to be a challenge.

Accomplishments that we're proud of

Accomplishments that we are proud of are that we were able to successfully implement both APIs into our backend. Implementation of a smooth user interface for the end-user. Especially when creating the Night Mode and the scrollers of the web extension. We were also very happy with the end product that we were able to come up with as well as the fact that we were able to complete the project within the time constraint.

What we learned

This project was a learning experience in AI ethics, user-centered design and real-world application of machine learning and Natural Language Processing Models. We learned about the complexities of developing a frontend application with the end-user in mind. We also learned about the dynamics of team collaboration in a high-stakes, time-constrained environment.

What's next for contentsift

Looking forward, we aim to expand ContentSift’s capabilities to more social media platforms such as Instagram, Facebook, Threads and even YouTube comment section. Furthermore, we want to include more languages and expand onto mobile apps as well. We plan to continuously refine our algorithms for better accuracy and fairness, and explore partnerships for wider deployment. Additionally, we want to train our own model as well to maximize the accuracy of our misinformation check. Moreover, in the future we will allow the user to control the severity of their content for moderation, as now any content that has a severity rating of greater than two is filtered. Lastly, we also want to expand into the moderation of image content for social media apps, helping users navigate through all forms of digital content safely and reliably.