Team Frog

LLM response showing the likelihood of the input being a scam + suggestions on how to combat the scam
List of victim stories + LLM suggestions on how to combat them
Sample screenshot of multimodal LLM input
RAG workflow including knowledge base used by LLM
Form section for submitting story
Upload section for submitting screenshot

Inspiration

Bank scams have resulted in significant financial losses for both the victims and the associated banks. In addition, bank scams (and scams in general) cause a great deal of emotional distress to the victims.

Scams usually exploit a lack of awareness or a heightened state of emotions in the victims to perform social engineering. While top-down approaches have been implemented by banks and the relevant authorities, there is a need for a bottom-up solution that provides an avenue for the victim to reactively consult while he is currently being scammed.

What it does

We believe that this will address the problem statement (Digital Threats: Tackling Financial Frauds) by:

Filling in the gaps in ever-evolving scam tactics employed by fraudsters by consistently updating our LLM and knowledge base
Provide a shorter reaction time for the victim, as banks and authorities still take some time to respond
Raise awareness for users by providing access to stories sent by other victims as well as suggestions to combat scams

From our website, users can submit their personal experiences or screenshots to the LLM to receive some information on:

Whether it is likely to be a scam
What kind of scam it is
Suggestions from the LLM on how to combat the scam

Users can also view stories submitted by other users, as well as the suggestions generated by the LLM.

While we are combating bank scams in particular, our app is flexible enough to support evaluations and suggestions for a variety of scams (such as impersonation scams, money mule scams, love scams etc). This is possible due to the extensive knowledge base that is used by the LLM with RAG.

How we built it

Using Selenium, we scrapped over 1700 scam stories from scamalert.sg, and fed this information into a knowledge base (ChromaDB + LangChain) for our LLM (OpenAI GPT-4O). We dockerised our NextJS frontend and FastAPI backend, and hosted the image on an Amazon EC2 instance.

For the deployment, we set up a reverse proxy with HTTPs configured to point to our app hosted on EC2.

What's next for Team Frog

We aim to obtain more scam stories for the knowledge base for our LLM, either via scraping our via some more structured datasets. This will improve the accuracy of our LLM, providing more informed outputs on the most recent scams.

One of the epics moving forward would be to add a chatbot for the user to interact with our LLM directly. This will allow the user to obtain help for a wider range of use cases, as well as to be able to clarify and prompt the LLM in a more nuanced manner.

We also aim to improve the stories page, by allowing users to search or filter for stories based on their category and related keywords. This will allow them to find out immediately if a related scam source (e.g. scammer's phone number or malicious links) has already been populated in our database.