ScamCall

an AI-agent that safeguards your communication by proactively blocking scam calls, ensuring every interaction is secure and trustworthy, protecting individuals and businesses from losses.

Comment 1

Inspiration

The idea for this project came from our encounters with financial fraud. While in Kenya, my colleagues and I became victims of M-Pesa scams, showing how scammers exploit individuals using social engineering. From fake transaction alerts to fraudulent customer service calls, we saw how these scams manipulate trust and urgency. Beyond our personal stories, fraudsters continue to prey on vulnerable populations like the elderly and small business proprietors, while businesses struggle to manage overwhelming customer inquiries. Scammers pose as customers or employees, deceiving people into revealing sensitive information.

Fraud detection systems like Truecaller and Hiya help identify spam calls but rely on crowdsourced data and blacklists, which scammers can bypass with new numbers. Google’s Call Screen filters unknown calls but doesn’t always detect social engineering used in scams. While these tools are helpful, they lack real-time fraud prevention for voice-based scams.

We wanted ScamCall to be a tool that screens unknown calls, helps spot suspicious ones, and makes handling support calls way easier.

By combining our personal experiences, we aim to make fraud prevention stronger, more accessible, and more effective in reducing financial fraud for individuals and businesses alike.

What it does

To address this, we created ScamCall—an AI-agent specifically designed to screen unfamiliar calls, identify suspicious ones, and assist businesses in managing customer support calls securely. When you use this on your device, you redirect all incoming calls to a separate number associated with our server.

Our system interacts with the Twilio API to determine if the phone number is flagged as suspicious. If an unknown number is detected, ScamCall’s AI-agent intercepts the call before it reaches you, interacts with the caller, and analyzes the conversation. Following this, you will receive an instant report alert through WhatsApp and on your ScamCall dashboard, where you can review the conversations and decide on the next steps.

How we built it

We developed our fraud detection system by combining various technologies for instant communication and AI-driven analysis. Initially, we built a wrapper around the OpenAI API, tailoring it with our unique dataset to identify fraud schemes specific to Africa. This customization enabled the AI to effectively scrutinize incoming calls for any suspicious activities. Subsequently, we set up a communication server using WebSockets to connect the Retell API for voice interactions and the Twilio API for phone number verification. This configuration allowed us to capture calls in real-time and direct them to the fraud detection model.

We incorporated a dual WebSocket server architecture to facilitate smooth communication between the fraud detection model and the voice interaction platform. This setup ensured that all incoming calls were analyzed immediately, with the AI agent assessing whether the call was fraudulent.

Lastly, we created an easy-to-use frontend dashboard where users can register, build their AI agent, and monitor all flagged calls they’ve received. The dashboard sends instant notifications and enables users to manage the results of identified calls effectively. This technical implementation guarantees a robust, real-time fraud detection system with seamless integration, equipping users with straightforward, intuitive tools to combat fraud.

Challenges we ran into

One of the major challenges we faced was the lack of publicly available datasets for training our fraud detection model. Most datasets focus on Western financial crimes, which don’t capture the unique fraud schemes common in Africa, such as 419 (advance fee fraud), SIM swap scams, family fraud, clinic emergency scams, government fraud, insurance scams, and mistaken transactions. Without this data, we had to build our dataset through researching real-world cases, analyzing user-reported incidents, and structuring the data for training. This process was time-consuming and posed difficulties in model accuracy due to the limited dataset.

Designing a fraud detection model that could adapt to evolving scam tactics was another challenge. Fraud patterns change quickly, so our model needed to identify new and emerging types of fraud. We tested various AI techniques, but one primary concern was managing the false positive rate. Identifying legitimate transactions as fraudulent would generate friction and diminish user trust in the system.

Moreover, establishing real-time communication between the caller and our AI agent presented a technical challenge. We developed a wrapper around the OpenAI API and fine-tuned it with our dataset, in addition to creating a communication server that integrated the Retell API for voice interactions and the Twilio API for phone number verification. However, latency issues during call interactions impacted the real-time fraud detection process, causing delays and affecting overall performance. Despite these obstacles, overcoming them enabled us to develop a working fraud detection system. These challenges emphasized the necessity for more open-source African fraud datasets to enhance innovation and strengthen financial security.

Accomplishments that we're proud of

A major accomplishment was developing an African-specific fraud detection system from scratch. With no existing datasets tailored to regional fraud schemes, we compiled and structured real-world cases like advance fee fraud, SIM swaps, and government scams into a usable machine-learning dataset. This effort ensured that our model addressed the unique fraud threats faced by African users. We fine-tuned the OpenAI GPT-3.5 Turbo base model, leveraging its natural language processing capabilities to enhance fraud detection. By applying machine learning techniques like anomaly detection and pattern recognition, the model can accurately flag fraudulent transactions while minimizing false positives, ensuring a smooth user experience.

The infrastructure we built includes a communication server that interacts with the Twilio API for phone number communication and the Retell API for voice interaction. We established a two-websocket server system to facilitate communication with our fine-tuned model, enabling real-time fraud detection. Additionally, we built a frontend dashboard that allows users to register, create an AI agent, and monitor all suspicious calls received.

This combination of dataset creation, fine-tuning, and infrastructure development ensures a robust, effective solution for detecting and preventing fraud in Africa.

What we learned

We learned there is a lack of data when it comes to fraud detection in Africa; that's why most of the solutions out there are not fit for the African market. Building our own dataset was a new experience for us. We also learned how crucial it is to strike the right balance between AI and human interaction. Tackling response time gave us a new respect for real-time applications.

What's next for ScamCall

Moving ahead, we intend to enhance our fraud detection system by collaborating with financial institutions, mobile money services, and fintech applications throughout Africa. Our objective is to create a real-time fraud detection API that businesses can use to thwart scams before they take place.

A key area of focus will be the expansion of our dataset. Given that fraud methods are continually changing, we plan to implement a crowdsourced reporting feature where users can report emerging scams, allowing us to dynamically update our model and improve detection precision. Another essential priority is to increase the model’s transparency. Numerous fraud detection systems function as opaque entities, leaving users in the dark about the reasons behind certain flagged transactions. We aim to incorporate explainable AI, offering clear explanations for each alert, which will foster trust among users and financial institutions.Furthermore, we are embedding fraud detection capabilities directly onto users' devices, allowing for immediate alerts regarding SMS scams, suspicious calls, and mobile transactions. With on-device AI, fraud detection can occur even when users are offline, enhancing accessibility and responsiveness.

In the end, our vision is to establish a dynamic, AI-based fraud prevention ecosystem that safeguards individuals and businesses from financial scams while evolving to meet new challenges.

Share this project:

Updates