Inspiration
The brand of Morgan & Morgan is represented by employees that are proud of representing their client and providing substantial information to them. We believe that one of Morgan & Morgan’s unique values is the focus on a personalized client experience. We chatted with multiple employees and we noticed a common thread of a less-than-optimal call experience. Employees waste time and are distracted by jotting down information during the call, cross-referencing relevant documents, which often are buried in emails or folders, and regaining context across multiple calls. CaseLink helps to solve this!
What it does
When a client calls in, they are prompted to say their name so that we can match their case by phone and name. This allows us to store a call transcript, pull up summaries of the client, and query for relevant documents with live links to the original source.
Unlike many projects at this hackathon, CaseLink seeks to augment existing tooling, rather than supplanting existing tooling entirely. Firms like Morgan & Morgan have existing processes and tooling for email, document sharing, and more. We try to integrate with existing infrastructure in a way that doesn’t disrupt existing operations and allows Morgan & Morgan to continue to use the tools they know and love.
How we built it
We used Twilio to intercept the call and create a conference call between the client, computer, and Morgan and Morgan representative. Twilio gives us the audio, which is transcribed into text which is then stored. The call gives us a phone number, and name which we use to do a database query to pull client data. We have a working keyword search for pulling up documents. Twilio data is connected via WebSockets and other endpoints are done with REST.
Challenges we ran into
- Tech Stack Pivot: We changed the front-end tech stack 12 hours into the hackathon. We went from React and Frammer-Motion to the Svelte framework.
- Twilio - The documentation for Twilio was not as clear so to understand the proper business logic needed for the project took a little time.
- Mailgun - We were unable to include Mailgun due to issues with their SDK. However, 3 minutes before Jordan & Rob created a method that was feasible however was not pushed to the main repo.
- Whisper - During the beginning of the project, Jordan & Rob were attempting to try to ingest byte arrays to create audio files however this became an issue due to not knowing how to create a reader that can ingest audio file bytes. We pivoted and utilized making a file that is generated after prompt/timing and this method ended up working flawlessly.
- SQLite Bugs - There were many typos in our SQLite business logic and internal dependency issues due to the fact that some team members were used to typescript.
Accomplishments that we're proud of
I am proud of our hacking of the audio stream coming from the phone calls. We had to deal with raw bytes in JavaScript, which is always a fun time. 🙂
Here is a list of what we finished from our task log:
- Whisper Transcriptions w/ OpenAI
- GPT Summarizations w/ Prompting
- Email Payloads
- Full Frontend functionality
- Document Text Search
- DOCX parsing
- Twilio SDK Usage
- CRUD operations
What did we learn?
- WebSockets
- Creating a better UI/UX experience with Svelte.
- Using Twilio SDK
- Prompting with GPT
What's next?
- Emails
- Summary with the top points of the call.
- Transcript of the call as a .txt file to the email.
- PDF Ingestion/Parsing
- New Client Ingestion
Built With
- express.js
- gpt4
- mammoth
- ngrock
- svelte
- tailwindscss
- twilio
- websockets
- whisper
Log in or sign up for Devpost to join the conversation.