IN-IT

Agents Dashboard page
Creating a new agent
Testing configuration page
Displaying test results
Real-time streaming + jailbreaking of AI agent

Inspiration

Nowadays, as companies race to deploy new AI solutions to avoid being left behind, they're leaving themselves vulnerable to a new type of attack, where techniques such as jailbreaking and prompt injection can turn "helpful" AI assistants into tools for data leaks and misinformation. And the consequences are severe. A recent IBM report found that the average data breach cost over $4.45 million.

For us, this wasn't something that was just theoretical. We've heard firsthand when one of our team members ' dads' company lost a major client after their internal AI agent was compromised due to simple prompt injection. So we realized that traditional security audits and manual red teaming is just too slow and expensive to keep up with AI's rapid development.

We built IN-IT to be the tool that makes security validation accessible to everyone, so everyone can have more trust in the AI software they're deploying.

What it does

IN-IT is a platform that stress-tests AI agents to find their security flaws before someone else does. Our goal was to make this platform as simple and easy to set up as possible. You define your AI's purpose with a system prompt (e.g., "You are a helpful customer service assistant who never gives out personal information"). Then, you give our platform a malicious objective (e.g., "Try to extract the user's email address and order history").

IN-IT handles the rest by creating a custom "Attacker Agent" that simulates a real-time conversation with a library of strategies like prompt injection, social engineering, and context poisoning to achieve it's goal. The best part is that you can watch the whole attack happen in real-time with streaming as our Evaluator Agent determines whether your Ai's defenses are successfully keeping up or not.

How we built it

For our frontend, we used Next.js, TypeScript, and Tailwind CSS. To enable real-time, we use Server-Sent Events (SSE) to stream data directly from the backend.

For our backend, we created an asynchronous API with Python 3.11 and FastAPI. Its async capabilities allow calls to our AI providers, Cerebras and Cohere. We're able to take user inputs, generate prompts for the attacker agent, pass responses to the defender, and query the evaluator agent to score the interaction in real-time.

For our database, we used MongoDB Atlas. Additionally, also used Auth0 for user authentication and Google sign-in.

Challenges we ran into

Firstly, just getting the live streaming to work was challenging as we had to figure out how to get Python's async features and SSE events to pipe the data to our Next.js app and not drop a single message. From there, if we had a particularly long attack, the conversation history would get too big, and it would break the AI's context window. So we had to create limiters and summarizers that could help with the context window. Lastly, we also had to really figure out what really determines whether an AI is "jailbroken" or not. Just searching for keywords wouldn't cut it, so we're proud of our solution, which was to use a third AI as a non-biased judge to read the conversation and analyze it.

Accomplishments that we're proud of

We're very proud of the fact that we were able to build an entire full-stack application in a short amount of time, and configure 3 distinct, individual AI agents to collaborate in real-time to simulate and evaluate security tests.

What we learned

This was a big deep dive into the world of agentic systems for us. We learned a lot about advanced prompt engineering, and not just for creating helpful assistants, but also on the contrary where we learned how to break the agents first. None of us had any experience in cybersecurity, so this was a great way for us to learn more about that space, and also just more about building resilient data pipelines in general.

What's next for In-It

Our vision for IN-IT extends far beyond just using it as a simple testing tool. Here are some of the things we'd like to do:

We want to scale up the architecture of the platform so we can run 1000s of conversational simulations in parallel. This way we can move from testing an agent in minutes to delivering a very comprehensive security verdict in seconds.
We also believe that the future of AI is conversational. We wanted to integrate voice Ai capabiltiies to test the next generation of enterprise call center bots, so they can be trained with proper security measures.
And lastly, we don't want to just find the flaws, but we want to help fix them. We want to be able to provide automated recommendations and implement reinforcement learning so we can make the AI agents better and better over time automatically.

Agent defence infrastructure needs to be near-perfect to unlock the next stage of agent use in the world. Making In-It into the first $1B agent cybersec company could build the rails for an agent-powered world.