ReCover

Generative AI got you covered when it comes to insurances

Comment

Inspiration

We want to simplify everyone’s lives, so that when an incident occurs they are not forced to go through pages of policies and terms and conditions.

What it does

We designed and implemented a data-driven approach to help insurance providers' customers to,

  • ease Q&A over large and complex insurance policy documents
  • identify if an claim situation is covered by customer's policy

leveraging Large-Language-Models (LLMs) and chatbot UX.

How we built it

Our approach contains four layers,

Infrastructure Layer

In this layer we provide generic LLM as capabilities to other layers, incl.,

  • ChatGPT 3.5 turbo
  • Llama2_7b_chat_hf_llm (deployed in Zurich Insurance's AWS environment)

Knowledge Layer

In this layer we collect un-structured insurance policies documents (provided by Zurich Insurance), and perform structured information discovery and extraction, to build a "Policy Knowledge Graph".

See an example policy graph here: https://github.com/noisythoughts/zurihack2023/blob/main/readme_diagrams/insurance_graph_example.png

LLM/GenAI Techniques

We're well-aware of the limitation of LLMs, such as hallucination. We build techniques such as,

  • retrieval augmented generation (RAG)
  • chain-of-thought
  • debate

to provide close to fact informations without losing the benefits of LLMs. These techniques can be easily scaled and further enhanced using additional human-based verification.

Challenges we ran into

We faced the following challenges,

  • lack of relevant domain knowledge: we're able to discuss and ran ideas with Zurich, eventually we came up with a "damage"-"cause"-"benefit"-"subject" taxonomy of insurance knowledge graph
  • no suitable AWS instances to deploy Llama2 LLM: we eventually identified a 13-billion model and suitable AWS instance, which is sufficient for our use-case
  • speeding up Q&A web interface: we wrote the frontend in Python, however the web interface is session dependent and often takes a lot of time to converting insurance policies into vector embeddings, we build in-memory cache to overcome this issue, therefore we're able to reduce the demo to 55 seconds
  • LLM token limit: we faced the input token limitation of LLM, we were able to leverage RAG technique, to first split documents into chunks, and when performing Q&A, we first identify the relevant chunks leveraging document similarity, and then for each of the chunk ask LLM to the given question, eventually re-combining the result

Accomplishments that we're proud of

We are proud of taking a structured approach working with unstructured data, thus reducing the freedom of expression of generative models and focus their power on specific tasks.

What we learned

We have discovered interesting problems that big businesses such as Zurich Insurance are facing. Allowing us to tackle and understand more in depth core challenges of generative AI, such as model hallucinations and the limitedness of the model context.

What's next for ReCover

  • Finalize offline "Policy Knowledge Graph" build
  • Tune queries used in Prompt Engineering
  • Extract inclusions / exclusions rules for coverage analysis, from all relevant documents.
  • Develop additional specialized agents for other types of queries (e.g. agent able to answer questions about billing)

Built With

Share this project:

Updates