Skip to content

Commit e772742

Browse files
alhendricksonalhendrickson
authored
feat(helm): Medcat service - support custom init container for model downloading (#51)
1 parent f63c22d commit e772742

File tree

8 files changed

+207
-60
lines changed

8 files changed

+207
-60
lines changed

‎deployment/kubernetes/charts/medcat-service-helm/README.md‎

Lines changed: 43 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -5,61 +5,76 @@ This Helm chart deploys the MedCAT service to a Kubernetes cluster.
55
## Installation
66

77
```sh
8-
helm install my-medcat-service oci://registry-1.docker.io/cogstacksystems/medcat-service-helm
8+
helm install medcat-service-helm oci://registry-1.docker.io/cogstacksystems/medcat-service-helm
99
```
1010

11+
## Usage
12+
For local testing, by default you can port forward the service using this command:
13+
14+
```sh
15+
kubectl port-forward svc/medcat-service-helm 5000:5000
16+
```
17+
18+
Then navigate to http://localhost:5000 to try the service. You can also use http://localhost:5000/docs to view the REST APIs
19+
20+
1121
## Configuration
1222

23+
To configure medcat service, create a values.yaml file and install with helm.
24+
25+
### Model Pack
1326
You should specify a model pack to be used by the service. By default it will use a small bundled model, which can be used for testing
1427

1528
---
16-
### Option 1: Use the demo model pack
29+
#### Default: Use the demo model pack
1730

1831
There is a model pack already bundled into medcat service, and is the default in this chart.
1932

2033
This pack is only really used for testing, and has just a few concepts built in.
2134

22-
### Option 2: Download Model on Startup
35+
#### Recommended: Download Model on Startup
2336

2437
Enable MedCAT to download the model from a remote URL on container startup.
2538

26-
Create a values file like `values-model-download.yaml` and update the env vars with:
39+
Create a values file like `values-model-download.yaml` and set these values:
2740
```yaml
28-
env:
29-
ENABLE_MODEL_DOWNLOAD: "true"
30-
MODEL_NAME: "medmen"
31-
MODEL_VOCAB_URL: "https://cogstack-medcat-example-models.s3.eu-west-2.amazonaws.com/medcat-example-models/vocab.dat"
32-
MODEL_CDB_URL: "https://cogstack-medcat-example-models.s3.eu-west-2.amazonaws.com/medcat-example-models/cdb-medmen-v1.dat"
33-
MODEL_META_URL: "https://cogstack-medcat-example-models.s3.eu-west-2.amazonaws.com/medcat-example-models/mc_status.zip"
34-
APP_MODEL_CDB_PATH: "/cat/models/medmen/cdb.dat"
41+
model:
42+
downloadUrl: "http://localhost:9000/models/my-model.zip"
43+
name: my-model.zip
3544
```
3645
3746
Use this if you prefer dynamic loading of models at runtime.
3847
39-
### Option 3: Get a model into a k8s volume, and mount it
48+
#### Advanced: Create a custom volume and load a model into it
4049
4150
The service can use a model pack if you want to setup your own download flow. For example, setup an initContainer pattern that downloads to a volume, then mount the volume yourself.
4251
43-
Use this env variable to point to the file:
52+
1. Create a persistent volume and PVC in kubernetes following the official documentation. Alternatively specifiy it in `values.extraManifests` and it will be created.
53+
54+
2. Create a values file like the following, which mounts the volume, and defines a custom init container.
4455

45-
Create a values file like `values-model-pack.yaml` and update the env vars with:
4656
```yaml
4757
env:
48-
# This defines the Model Pack used by the medcat service
49-
APP_MEDCAT_MODEL_PACK: "/cat/models/examples/example-medcat-v1-model-pack.zip"
50-
```
58+
APP_MEDCAT_MODEL_PACK: "/my/models/custom-model.zip"
59+
volumeMounts:
60+
name: model-volume
61+
mountPath: /my/models
62+
63+
volumes:
64+
- name: model-volume
65+
persistentVolumeClaim:
66+
claimName: my-custom-pvc
67+
extraInitContainers:
68+
- name: model-downloader
69+
image: busybox:1.28
70+
# In this command, you can write custom code required to download a file. For example you could configure authentication.
71+
command: ["sh", "-c", "wget -O /my/models/custom-model.zip http://example.com"]
72+
volumeMounts:
73+
- name: model-volume
74+
mountPath: /my/models
5175
52-
## Example
53-
54-
```sh
55-
helm install my-medcat ./medcat-chart -f values-model-pack.yaml
5676
```
5777

58-
or
59-
60-
```sh
61-
helm install my-medcat ./medcat-chart -f values-model-download.yaml
62-
```
6378

6479
### DeID Mode
6580

@@ -73,7 +88,7 @@ env:
7388
```
7489
7590
76-
## GPU Support
91+
### GPU Support
7792
7893
To run MedCAT Service with GPU acceleration, use the GPU-enabled image and set the pod runtime class accordingly.
7994
@@ -101,7 +116,7 @@ env:
101116
> - [NVIDIA GPU Feature Discovery](https://github.com/NVIDIA/gpu-feature-discovery)
102117
> - The [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/)
103118
104-
### Test GPU support
119+
#### Test GPU support
105120
You can verify that the MedCAT Service pod has access to the GPU by executing `nvidia-smi` inside the pod.
106121

107122

‎deployment/kubernetes/charts/medcat-service-helm/ci/default-values.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Empty values file to run CI tests against the defaults

‎deployment/kubernetes/charts/medcat-service-helm/ci/initcontainer-values.yaml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# This values file is used to test the init container functionality
2+
# It shows using an initContainer and a volume to run custom code to provide a model pack
3+
env:
4+
APP_MEDCAT_MODEL_PACK: "/my/models/custom-model.zip"
5+
volumeMounts:
6+
- name: model-volume
7+
mountPath: /my/models
8+
volumes:
9+
- name: model-volume
10+
persistentVolumeClaim:
11+
claimName: medcat-model-pvc
12+
13+
extraInitContainers:
14+
- name: custom-init-container
15+
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
16+
# For demo purposes - copy the exmple model pack and reference under new path.
17+
# This demonstrates that we can run any custom code desired to get a model pack.
18+
command:
19+
[
20+
"sh",
21+
"-c",
22+
"cp /cat/models/examples/example-medcat-v2-model-pack.zip /my/models/custom-model.zip",
23+
]
24+
volumeMounts:
25+
- name: model-volume
26+
mountPath: /my/models
27+
28+
extraManifests:
29+
- apiVersion: v1
30+
kind: PersistentVolumeClaim
31+
metadata:
32+
name: medcat-model-pvc
33+
spec:
34+
accessModes:
35+
- ReadWriteOnce
36+
resources:
37+
requests:
38+
storage: 100Mi

‎deployment/kubernetes/charts/medcat-service-helm/templates/deployment.yaml‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -79,15 +79,17 @@ spec:
7979
{{- if or .Values.model.downloadUrl .Values.volumeMounts }}
8080
volumeMounts:
8181
{{- if .Values.volumeMounts }}
82-
{{- toYaml .Values.volumeMounts | nindent 2 }}
82+
{{- toYaml .Values.volumeMounts | nindent 12 }}
8383
{{- end }}
8484
{{- if .Values.model.downloadUrl }}
8585
- name: models
8686
mountPath: /models
8787
{{- end }}
8888
{{- end }}
89-
{{- if .Values.model.downloadUrl }}
89+
{{- if or .Values.extraInitContainers .Values.model.downloadUrl }}
9090
initContainers:
91+
{{- end }}
92+
{{- if .Values.model.downloadUrl }}
9193
- name: model-downloader
9294
image: busybox:1.28
9395
command:
@@ -111,11 +113,14 @@ spec:
111113
- name: models
112114
mountPath: /models
113115
{{- end }}
114-
116+
{{- $root := . -}}
117+
{{- with .Values.extraInitContainers }}
118+
{{- tpl (toYaml .) $root | nindent 8 }}
119+
{{- end }}
115120
{{- if or .Values.model.downloadUrl .Values.volumes }}
116121
volumes:
117122
{{- if .Values.volumes }}
118-
{{- toYaml .Values.volumes | nindent 2 }}
123+
{{- toYaml .Values.volumes | nindent 8 }}
119124
{{- end }}
120125
{{- if .Values.model.downloadUrl }}
121126
- name: models

‎deployment/kubernetes/charts/medcat-service-helm/templates/extraManifests.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{{ range .Values.extraManifests }}
2+
---
3+
{{ tpl (toYaml .) $ }}
4+
{{ end }}

‎deployment/kubernetes/charts/medcat-service-helm/values.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,3 +233,9 @@ networkPolicy:
233233
# app.kubernetes.io/name: model-downloader
234234
# ports:
235235
# - port: 5000
236+
237+
# Additional init containers to run before the main container. Can be templated
238+
extraInitContainers: []
239+
240+
# Additional manifests to deploy to kubernetes. Can be templated
241+
extraManifests: []

‎docs/docs/platform/deployment/helm/charts/medcat-service-helm.md‎

Lines changed: 89 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -5,58 +5,125 @@ This Helm chart deploys the MedCAT service to a Kubernetes cluster.
55
## Installation
66

77
```sh
8-
helm install my-medcat-service oci://registry-1.docker.io/cogstacksystems/medcat-service-helm
8+
helm install medcat-service-helm oci://registry-1.docker.io/cogstacksystems/medcat-service-helm
99
```
1010

11+
## Usage
12+
13+
For local testing, by default you can port forward the service using this command:
14+
15+
```sh
16+
kubectl port-forward svc/medcat-service-helm 5000:5000
17+
```
18+
19+
Then navigate to http://localhost:5000 to try the service. You can also use http://localhost:5000/docs to view the REST APIs
20+
1121
## Configuration
1222

23+
To configure medcat service, create a values.yaml file and install with helm.
24+
25+
### Model Pack
26+
1327
You should specify a model pack to be used by the service. By default it will use a small bundled model, which can be used for testing
1428

1529
---
16-
### Option 1: Use the demo model pack
30+
31+
#### Default: Use the demo model pack
1732

1833
There is a model pack already bundled into medcat service, and is the default in this chart.
1934

20-
This pack is only really used for testing, and has just a few concepts built in.
35+
This pack is only really used for testing, and has just a few concepts built in.
2136

22-
### Option 2: Download Model on Startup
37+
#### Recommended: Download Model on Startup
2338

2439
Enable MedCAT to download the model from a remote URL on container startup.
2540

26-
Create a values file like `values-model-download.yaml` and update the env vars with:
41+
Create a values file like `values-model-download.yaml` and set these values:
42+
2743
```yaml
28-
env:
29-
ENABLE_MODEL_DOWNLOAD: "true"
30-
MODEL_NAME: "medmen"
31-
MODEL_VOCAB_URL: "https://cogstack-medcat-example-models.s3.eu-west-2.amazonaws.com/medcat-example-models/vocab.dat"
32-
MODEL_CDB_URL: "https://cogstack-medcat-example-models.s3.eu-west-2.amazonaws.com/medcat-example-models/cdb-medmen-v1.dat"
33-
MODEL_META_URL: "https://cogstack-medcat-example-models.s3.eu-west-2.amazonaws.com/medcat-example-models/mc_status.zip"
34-
APP_MODEL_CDB_PATH: "/cat/models/medmen/cdb.dat"
44+
model:
45+
downloadUrl: "http://localhost:9000/models/my-model.zip"
46+
name: my-model.zip
3547
```
3648
3749
Use this if you prefer dynamic loading of models at runtime.
3850
39-
### Option 3: Get a model into a k8s volume, and mount it
51+
#### Advanced: Create a custom volume and load a model into it
4052
4153
The service can use a model pack if you want to setup your own download flow. For example, setup an initContainer pattern that downloads to a volume, then mount the volume yourself.
4254
43-
Use this env variable to point to the file:
55+
1. Create a persistent volume and PVC in kubernetes following the official documentation. Alternatively specifiy it in `values.extraManifests` and it will be created.
56+
57+
2. Create a values file like the following, which mounts the volume, and defines a custom init container.
4458

45-
Create a values file like `values-model-pack.yaml` and update the env vars with:
4659
```yaml
4760
env:
48-
# This defines the Model Pack used by the medcat service
49-
APP_MEDCAT_MODEL_PACK: "/cat/models/examples/example-medcat-v1-model-pack.zip"
61+
APP_MEDCAT_MODEL_PACK: "/my/models/custom-model.zip"
62+
volumeMounts:
63+
name: model-volume
64+
mountPath: /my/models
65+
66+
volumes:
67+
- name: model-volume
68+
persistentVolumeClaim:
69+
claimName: my-custom-pvc
70+
extraInitContainers:
71+
- name: model-downloader
72+
image: busybox:1.28
73+
# In this command, you can write custom code required to download a file. For example you could configure authentication.
74+
command:
75+
["sh", "-c", "wget -O /my/models/custom-model.zip http://example.com"]
76+
volumeMounts:
77+
- name: model-volume
78+
mountPath: /my/models
5079
```
5180

52-
## Example
81+
### DeID Mode
82+
83+
The service can perform DeID of EHRs by swithcing to the following values
5384

54-
```sh
55-
helm install my-medcat ./medcat-chart -f values-model-pack.yaml
5685
```
86+
env:
87+
APP_MEDCAT_MODEL_PACK: "/cat/models/examples/example-deid-model-pack.zip"
88+
DEID_MODE: "true"
89+
DEID_REDACT: "true"
90+
```
91+
92+
### GPU Support
93+
94+
To run MedCAT Service with GPU acceleration, use the GPU-enabled image and set the pod runtime class accordingly.
95+
96+
Note GPU support is only used for deidentification
97+
98+
Create a values file like `values-gpu.yaml` with the following content:
99+
100+
```yaml
101+
image:
102+
repository: ghcr.io/cogstack/medcat-service-gpu
57103
58-
or
104+
runtimeClassName: nvidia
105+
106+
resources:
107+
limits:
108+
nvidia.com/gpu: 1
109+
env:
110+
APP_CUDA_DEVICE_COUNT: 1
111+
APP_TORCH_THREADS: -1
112+
DEID_MODE: true
113+
```
114+
115+
> To use GPU acceleration, your Kubernetes cluster should be configured with the NVIDIA GPU Operator or the following components:
116+
>
117+
> - [NVIDIA device plugin for Kubernetes](https://github.com/NVIDIA/k8s-device-plugin)
118+
> - [NVIDIA GPU Feature Discovery](https://github.com/NVIDIA/gpu-feature-discovery)
119+
> - The [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/)
120+
121+
#### Test GPU support
122+
123+
You can verify that the MedCAT Service pod has access to the GPU by executing `nvidia-smi` inside the pod.
59124

60125
```sh
61-
helm install my-medcat ./medcat-chart -f values-model-download.yaml
126+
kubectl exec -it <POD_NAME> -- nvidia-smi
62127
```
128+
129+
You should see the NVIDIA GPU device listing if the GPU is properly accessible.

0 commit comments

Comments
 (0)