Ez Form

An AI-powered voice assistant that helps people complete forms just by talking.

Comment

Our Mission 🐝

We set out to apply AI in a way that genuinely improves people’s lives. Ez Form uses conversational AI and voice technology to remove barriers created by keyboards, paperwork, and complex forms, especially for older adults and anyone who struggles with traditional digital interfaces. Our mission is to make essential tasks more accessible, human, and stress-free through AI.

Inspiration 💥

We wanted to build something for older adults and anyone who struggles with keyboards or dense paperwork. Forms are frustrating and inaccessible, so we built a voice-first experience that lets people fill PDFs just by talking, like having someone sit beside you and help you through the process.

What it does 🦾📋

Ez Form lets users upload a PDF form, then have a voice conversation to fill it out. The app extracts form fields, asks questions aloud, listens to spoken responses, tracks completion, and generates a filled PDF for download. The result is a completed form without ever touching a keyboard.

How we built it 🧱

The backend uses FastAPI to handle file uploads, sessions, and PDF field extraction and filling with PyPDF2. Google Gemini powers the conversational flow, guiding users dynamically through form fields and adapting to spoken input in real time. The frontend is React with TypeScript, using the Web Speech API for speech recognition and speech synthesis. The overall flow is upload -> conversation -> progress tracking -> download.

Challenges we ran into ⚔️

  • Speech Recognition Latency: Reducing delay between user speech and system responses to maintain a natural conversational flow.
  • Browser Compatibility: Handling inconsistencies in microphone access and Web Speech API behavior across different browsers.
  • AI-to-Form Field Alignment: Ensuring spoken responses were reliably mapped to the correct PDF form fields.
  • Microphone Permissions & UI Responsiveness: Managing audio permissions and async processing without blocking or freezing the interface.

Accomplishments that we're proud of 👏

  • End-to-End Product: Built a complete workflow from PDF upload to downloadable, filled forms.
  • AI Accuracy: Designed reliable AI-driven field extraction and response mapping to ensure correct and consistent form completion.
  • Accessibility-First Design: Created a voice-first interface that reduces reliance on keyboards and complex UI interactions.
  • Practical AI Application: Demonstrated how conversational AI can move beyond demos to reduce real-world friction in everyday tasks.

What we learned 🧠

We learned how to integrate AI into a real product, the practical limits of speech recognition, and how to extract and fill PDF form fields across different file types.

What's next for Ez Form 🔮

Next we want smarter field matching, faster and more reliable voice responsiveness, multilingual support, and a better review/edit step before final download. Long term, we see Ez Form as a foundation for making any form-based process accessible through natural conversation.

Built With

Share this project:

Updates