Dial In: AI-powered interviews & insights.

We've developed an AI agent to automate interviews. The agent conducts interviews on predefined topics, processes the conversation post-call, and generates insights across the entire interview corpus.

Comment

Inspiration

From personal experience, we understood how useful user interviews are to learn their needs and preferences. However, scaling this approach is challenging due to the time and costs involved. Some pain points in running user interviews include:

  • Coordinating schedules for interviewees and interviewers
  • Time taken to conduct interviews
  • Time-consuming analysis of transcripts to identify themes

Motivated by these challenges, we sought to create a solution that simplifies the process of conducting user interviews and extracting valuable insights. We want to make it easier for teams to gather meaningful information while reducing time and resource constraints.

Beyond simplifying user interviews, Dial In can be useful in various industries and applications, such as market research, journalism, and academic research, enabling more efficient data collection and insightful decision-making.

What it does

We developed an AI agent that conducts interviews on behalf of the requesting team to achieve their specific objectives (e.g., Objective: Conduct an interview to understand how users are utilizing ChatGPT and the pain points they encounter). The AI agent intelligently generates questions during the call, adapting to the conversation to fulfill the objective. Upon completion of the interview, the following steps are carried out:

  • Transcription: The call is transcribed into text format for further processing.
  • Semantic Processing: The transcribed content is broken down into semantically relevant chunks, vectorized, and categorized based on identified topics.
  • Querying and Insight Generation: When the requesting team poses questions about the interview corpus (e.g., What do users feel about the membership?), we retrieve relevant text chunks and feed them into OpenAI to synthesize an answer

How We Built It

  • OpenAI: For generating insights and embeddings
  • Pinecone: Vector database for efficient data management
  • Deepgram: Speech-to-text conversion for transcription
  • Eleven Labs: Text-to-speech synthesis for AI agent's voice
  • Vocode: Voice Code Orchestrator to streamline the process
  • Vercel Hosting and Deployment

Challenges we ran into

  • Fine-tuning instructions to the AI agent to make the interview process feel natural and engaging.
  • Fine-tuning instructions to the AI-agent to ensure it doesn't veer off topic but still remains flexible
  • Orchestrating the voice call to ensure the speaker doesn't unintentionally override the AI-interviewer
  • Deploying the voice call app from Replit.

Accomplishments that we're proud of

  • Building an end-to-end AI interview solution that streamlines the entire process of conducting interviews, transcribing, and analyzing the data.
  • Developing an AI agent capable of conducting natural and engaging interviews, adapting its questions in real-time to fulfill predefined objectives.

What we learned

  • How to orchestrate AI-led voice calls
  • How to work with different speech related technologies (e.g., Eleven Labs, Deepgram)

What's Next for Dial In: AI-Powered Interviews & Insights

  1. Expand AI capabilities: Refine the AI agent's abilities to handle complex interviews, topics, and questions.
  2. Additional language support: Integrate multilingual capabilities for global accessibility.
  3. Industry-specific adaptations: Develop tailored solutions for specific sectors like market research, journalism, HR, or customer feedback.

Built With

  • deepgram
  • eleven-labs
  • openai
  • pinecone
  • vercel
  • vocode
Share this project:

Updates