Be Square

Inspiration ✨

We were inspired by how long it takes to create diagrams and the such in Adobe Express. We wanted to create a solution that would help streamline the process of adding elements into the canvas, hands free.

What it does 🤔

Our Adobe Express add-on, is a virtual assistant, named Doug, that can take voice commands or text in natural language and change or add elements to a page.

How we built it 🤖

Doug is powered by OpenAI's whisper model and gpt-4o to interpret natural language. It is run off a backend node.js server that interacts with the add-on through fetch and websockets. Doug uses function calling to create and modify creative elements in Adobe Express.

Challenges we ran into 😡

Adobe Express doesn't allow external modules on the client side. We were trying to implement webpack, since the error message told us that it was necessary, but it was not the solution. We spent a large amount of time simply trying to get the add-on to be able to call openai functions.

another challenge was that the node.js server had to be able to call functions in the front-end add-on, resulting in the use of websockets, which none of us had any experience in and took a large amount of time to complete.

Accomplishments that we're proud of 🫡

All of our basic functions and backend are functional, and Doug is able to create diagrams and creative elements. It was a massive hurdle to get to this point, but we are proud of our accomplishments to have had a functional project at the end, since none of us were sure if this was possible or not.

What we learned😎

Although we have heard of the idea, we were all new to working with AI implementation, especially in a node.js server. We didn't think that it would be as windy of a road as it had been for us to implement our functions, since with every error and stuff simply not working, we were forced to go through different hoops and rings to get to the solution.

What's next for Be Square?⭐

Doug can theoretically work with a much larger number of functions, giving him more and more options to be able to help the user. In the future, Doug could even interact with other Adobe Express add-ons to have even greater creative options for the user.