Dataset Information:

Created By: https://github.com/WengLab-InformaticsResearch/CHIA

Article: https://www.nature.com/articles/s41597-020-00620-0

Data: https://figshare.com/articles/dataset/Chia_Annotated_Datasets/11855817

OMOP CDM: https://ohdsi.github.io/CommonDataModel/

Interactive ODHSI DB: https://athena.ohdsi.org/search-terms/terms?domain=Measurement

DB Schema: https://ohdsi.github.io/CommonDataModel/cdm53.html

Preprocessing Help:

https://github.com/uf-hobi-informatics-lab/ClinicalTransformerNER

Transformer models with named entity recognition in clinical trial eligibility criteria

https://github.com/ctgatecci/Clinical-trial-eligibility-criteria-NER

Paper: https://dl.acm.org/doi/pdf/10.1145/3459930.3469560

Knowledge Graph:

https://www.kaggle.com/code/pavansanagapati/knowledge-graph-nlp-tutorial-bert-spacy-nltk/notebook

Saving and Loading Pytorch Models

----------------------------------

https://pytorch.org/tutorials/beginner/saving_loading_models.html

https://github.com/umar1997/MuBAF/blob/main/BIDAF/BiDAF_Implementation.ipynb

https://github.com/umar1997/MBZUAI/blob/main/PyTorch%20(Udemy-HF-Aladdin_Pearson)/Udemy/3.%20Artificial-Neural-Networks/7.%20Recap-Saving-and-Loading-Trained-Models-NO_VIDEO.ipynb

SUMMARY GENERATION WITH T5

-----------------------

https://colab.research.google.com/github/abhimishra91/transformers-tutorials/blob/master/transformers_summarization_wandb.ipynb#scrollTo=1qOEliuKrBqY

https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/summarization.ipynb#scrollTo=EZorj0CZpPzQ

Fine Tuning

------------

FineTuning WikiSQL

https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb

https://github.com/amazon-research/nl2sql-omop-cdm/blob/main/src/engine/step4/model_dev/t5_training.py

https://huggingface.co/mrm8488/t5-base-finetuned-wikiSQL?text=translate+English+to+SQL%3A+How+many+models+were+finetuned+using+BERT+as+base+model%3F

FineTuning Summarization

https://towardsdatascience.com/fine-tuning-a-t5-transformer-for-any-summarization-task-82334c64c81

NATURAL LANGUAGE TO SQL

-----------------------

https://towardsdatascience.com/natural-language-to-sql-use-it-on-your-own-database-d4cd5784d081

- [Text-to-SQL] Learning to query tables with natural language

https://towardsdatascience.com/text-to-sql-learning-to-query-tables-with-natural-language-7d714e60a70d

- Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions

- EditSQL

https://arxiv.org/abs/1909.00786

- A Comparative Survey of Recent Natural Language Interfaces for Databases

https://arxiv.org/pdf/1906.08990.pdf

- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

https://arxiv.org/pdf/1911.04942.pdf

-IR Net + BERT

- Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation

https://arxiv.org/pdf/1905.08205.pdf

NAMED ENTITY RECOGNITION

-------------------------

https://towardsdatascience.com/named-entity-recognition-with-bert-in-pytorch-a454405e0b6a

https://github.com/umar1997/MBZUAI/blob/main/NLP%20702/Assignments/2.%20Named%20Entity%20Recognition%20(BERT)/Named_Enitity_Recognition_with_BERT.ipynb

Custom Named Entity Recognition with BERT (Similar to my repo above)

https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/BERT/Custom_Named_Entity_Recognition_with_BERT_only_first_wordpiece.ipynb#scrollTo=deLB9HVX5I6F

https://www.kaggle.com/code/jonofields/bert-ner-sentence/notebook

Adding custom layer on top of a hugging face

https://towardsdatascience.com/adding-custom-layers-on-top-of-a-hugging-face-model-f1ccdfc257bd

Token Classification Hugging Face

https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/token_classification.ipynb#scrollTo=n9qywopnIrJH

NER EVALUATION METRICS

https://www.davidsbatista.net/blog/2018/05/09/Named_Entity_Evaluation/#:~:text=When%20you%20train%20a%20NER%20system%20the%20most,evaluate%20with%20metrics%20at%20a%20full%20named-entity%20level.

MASKED LANGAUGE MODELLING

--------------------------

https://discuss.huggingface.co/t/how-to-train-your-own-corpus-without-labels/6369

https://huggingface.co/docs/transformers/training

https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/language_modeling.ipynb#scrollTo=HFASsisvIrIb

https://github.com/huggingface/transformers/tree/main/examples/pytorch

https://towardsdatascience.com/masked-language-modelling-with-bert-7d49793e5d2c

https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/chapter7/section3_pt.ipynb#scrollTo=sNa2Lr9WifWX

https://morioh.com/p/814ec71da7ff

https://discuss.huggingface.co/t/roberta-mlm-fine-tuning/1330/2 , https://github.com/huggingface/transformers/issues/2691

https://www.youtube.com/results?search_query=MLM+for+fine+tuning , https://www.youtube.com/watch?v=5Bi3vQE2Hak&ab_channel=TheArtificialGuy

https://huggingface.co/docs/transformers/tasks/language_modeling

QUERY GENERATION

-----------------

Encoder-decoder TREQS model (from MIMIC): Text-to-SQL Generation for Question Answering on Electronic Medical Records (vt.edu)

https://people.cs.vt.edu/~reddy/papers/WWW20.pdf

GitHub Repository: https://github.com/wangpinggl/TREQS

MedTS paper (which compared favorably to TREQS): A BERT-Based Generation Model to Transform Medical Texts to SQL Queries for Electronic Medical Records: Model Development and Validation - PMC (nih.gov)

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8701710/

Github Repository: https://github.com/pan915/MedTS

T5 model finetuned with wikiSQL (https://huggingface.co/mrm8488/t5-small-finetuned-wikiSQL)

GitHub Repository: https://github.com/amazon-research/nl2sql-omop-cdm

GitDetail: https://www.gitdetail.com/repositories/amazon-research/nl2sql-omop-cdm/3410

Data (text-and-sql pairs based on the OMOP)

GitHub Repository: https://github.com/OHDSI/Nostos

WORD EMBEDDINGS

----------------

BERT

https://towardsdatascience.com/3-types-of-contextualized-word-embeddings-from-bert-using-transfer-learning-81fcefe3fe6d

BiBERT

https://github.com/dmis-lab/biobert-pytorch

https://huggingface.co/dmis-lab/biobert-base-cased-v1.2

Paper: https://arxiv.org/ftp/arxiv/papers/1901/1901.08746.pdf

MedBERT

https://github.com/ZhiGroup/Med-BERT

Paper: https://arxiv.org/ftp/arxiv/papers/2005/2005.12833.pdf

Distance/Cosine Similarity For Embeddings:

https://github.com/arushiprakash/MachineLearning/blob/main/BERT%20Word%20Embeddings.ipynb

COMPLEX QA USING KNOWLEDGE GRAPHS

---------------------------------

Mikhail Galkin - Complex Question Answering over Knowledge Graphs

https://www.youtube.com/watch?v=hjCO_7nSqzA

https://github.com/heathersherry/Knowledge-Graph-Tutorials-and-Papers/blob/master/topics/Knowledge%20Graph%20Question%20Answering%20(KGQA).md

- Complex Knowledge Base Question Answering: A Survey 2021

https://arxiv.org/pdf/2108.06688v1.pdf

- A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges

https://arxiv.org/pdf/2007.13069.pdf

- Question Answering over Knowledge Graphs with Neural Machine Translation and Entity Linking

https://arxiv.org/abs/2107.02865

TEMPLATE BASED QA

---------------------------------

- Template-Based Question Answering over Linked Data using Recursive Neural Networks (Thesis)

https://core.ac.uk/download/pdf/195379916.pdf

- A Guided Template-Based Question Answering System over Knowledge Graphs

http://ceur-ws.org/Vol-2262/ekaw-demo-20.pdf

- Template-based Question Answering using Recursive Neural Networks

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9364639

EXTRA INFORMATION

------------------

# Negation Detection

NegBERT:

https://github.com/adityak6798/Transformers-For-Negation-and-Speculation/blob/master/Transformers_for_Negation_and_Speculation.ipynb

HuggingFace = https://huggingface.co/bvanaken/clinical-assertion-negation-bert?text=The+patient+recovered+during+the+night+and+now+denies+any+%5Bentity%5D+shortness+of+breath+%5Bentity%5D.

# Fine Tuning Pretrained Models with Hugging Face

https://towardsdatascience.com/fine-tuning-pretrained-nlp-models-with-huggingfaces-trainer-6326a4456e7b

# Criteria2Query https://github.com/OHDSI/Criteria2Query

# Text2SQL

https://towardsdatascience.com/text-to-sql-learning-to-query-tables-with-natural-language-7d714e60a70d

https://towardsdatascience.com/natural-language-to-sql-use-it-on-your-own-database-d4cd5784d081

https://paperswithcode.com/task/text-to-sqls

# Models/Papers

https://paperswithcode.com/paper/content-enhanced-bert-based-text-to-sql

https://paperswithcode.com/paper/rat-sql-relation-aware-schema-encoding-and-1

https://paperswithcode.com/paper/sqlnet-generating-structured-queries-from

# Text-to-SQL # Generation for Question Answering on Electronic Medical Record

https://github.com/wangpinggl/TREQS chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://dmkd.cs.vt.edu/papers/WWW20.pdf

# WikiSQL Train Data

https://github.com/dadashkarimi/seq2sql/blob/master/data/wikisql/wikisql_train1000.tsv