This project implements an AI agent that utilizes Retrieval-Augmented Generation (RAG) to access the full documentation of Ableton Live 12. The agent can efficiently retrieve and provide information from the documentation based on user queries. It also uses web scraping to get the latest information from the documentation.

The AI agent is designed to read and process the Ableton Live 12 documentation, allowing users to ask questions and receive accurate responses based on the official documentation. The project leverages embeddings to enhance the retrieval process, ensuring that the agent can quickly access relevant information.

• RAG Implementation: Combines retrieval and generation techniques to provide accurate responses.
• Embeddings: Each URL from the Ableton Live 12 documentation is stored as a resource in the database, with corresponding embeddings for efficient retrieval.
• Multimodal: The agent is capable to interpret images on the documentation and provide relevant images on the response.
• Web Scraping: The webscrapping script is available at ./scripts/scrap.ts.

• Node.js (version >= v18.17.0)
• npm or yarn
• A database (e.g., PostgreSQL) for storing resources and embeddings
• A Firecrawl API key (https://www.firecrawl.dev/)
• A OpenAI API key (https://openai.com/index/openai-api/)
• A Google Generative AI API key (https://cloud.google.com/ai/generative-ai)

To run the application, you need to create a .env file in the root directory of the project with the following environment variables:

DATABASE_URL=<your_postgresql_database_url>
OPENAI_API_KEY=<your_openai_api_key>
FIRECRAWL_API_KEY=<your_firecrawl_api_key>
GOOGLE_GENERATIVE_AI_API_KEY=<your_google_generative_ai_api_key>

• DATABASE_URL: The connection string for your PostgreSQL database.
• OPENAI_API_KEY: Your API key for accessing OpenAI services for chat and embeddings.
• FIRECRAWL_API_KEY: The API key for the Firecrawl service used for web scraping.
• GOOGLE_GENERATIVE_AI_API_KEY: The API key for the Google Generative AI service used for image description.

1. Clone the repository:

   git clone https://github.com/Saraceni/AbletonAiAgent
   cd your-repo-name

2. Install dependencies:

   pnpm install
   # or
   yarn install

3. Set up your database and configure the connection settings in your environment variables.

1. Run the Application:

   pnpm run dev
   # or
   yarn dev

2. Run the web scraping script:

   • The web scraping API is available at ./scripts/script.
   • First run: The GET endpoint reads urls.md, extracts text content from each page, and creates text embeddings for each resource in the database.
   • Second run: The GET endpoint processes all images from previously scraped pages, creates descriptions for them using Google's Generative AI, and stores these as media resources in the database.
   
   

   pnpm run scrap
   # or
   yarn scrap

   [!TIP] Remember to add the environmental variables so you can run the script without errors

Each resource in the database represents a URL from the Ableton documentation. The schema includes:

• Resource ID: Unique identifier for each resource.
• URL: The URL of the Ableton documentation.
• Embeddings: Vector representations of the resource for efficient retrieval.
• Media: Descriptions of images scraped from the documentation.

Contributions are welcome! Please open an issue or submit a pull request for any enhancements or bug fixes.

This project is licensed under the MIT License - see the LICENSE file for details.

• Ableton for their excellent documentation.
• OpenAI for the assistant.

This is the starter project for the Vercel AI SDK Retrieval-Augmented Generation (RAG) guide.

In this project, you will build a chatbot that will only respond with information that it has within its knowledge base. The chatbot will be able to both store and retrieve information. This project has many interesting use cases from customer support through to building your own second brain!

This project will use the following stack:

• Next.js 14 (App Router)
• Vercel AI SDK
• OpenAI
• Drizzle ORM
• Postgres with pgvector
• shadcn-ui and TailwindCSS for styling