Skip to content

Commit fe39b6a

Browse files
validbeckvalidbeck
authored
Updated workflows/actions to dynamically generate an .env file (#552)
* Updated store-credentials-in-env-file.qmd * Added Enable monitoring edge case * Pulled in the working tests from beck/no-changed-notebooks * Duplicated workflow & actions to staging & prod * Updating workflows to check against the correct branches * Adding Make command * Retrieving latest from validmind-library
1 parent fac548b commit fe39b6a

62 files changed

Lines changed: 946 additions & 93446 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
OPENAI_API_KEY=ADD_YOUR_KEY_HERE
22

33
# API INFO TO RUN /notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
4-
PLATFORM_API_KEY=ADD_YOUR_API_KEY_HERE
5-
PLATFORM_API_SECRET=ADD_YOUR_API_SECRET_HERE
4+
VM_API_HOST=<api_host>
5+
VM_API_KEY=<api_key>
6+
VM_API_SECRET=<api_secret>
7+
VM_API_MODEL=<model>

‎.github/actions/demo-notebook/action.yml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
name: Execute demo notebook
22
description: Installs python3, validmind, checks dependencies then executes ONLY the Intro for Model Developers notebook with development heap tracking
33

4-
inputs: {}
4+
inputs:
5+
env_file:
6+
description: "Load the created .env file"
7+
required: true
58

69
runs:
710
using: "composite"
@@ -26,10 +29,21 @@ runs:
2629
pip install pycairo
2730
pip check
2831
32+
- name: Ensure .env file is available
33+
shell: bash
34+
id: find_env
35+
run: |
36+
if [ ! -f "${{ inputs.env_file }}" ]; then
37+
echo "Error: .env file not found at ${{ inputs.env_file }}"
38+
exit 1
39+
fi
40+
2941
- name: Execute ONLY the Intro for Model Developers notebook with heap development
3042
shell: bash
43+
if: ${{ steps.find_env.outcome == 'success' }}
3144
run: |
3245
cd site
46+
source ../${{ inputs.env_file }}
3347
quarto render --profile exe-demo notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb &> render_errors.log || {
3448
echo "Execute for intro_for_model_developers_EXECUTED.ipynb failed";
3549
cat render_errors.log;

‎.github/actions/prod-notebook/action.yml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
name: Execute prod notebook
22
description: Installs python3, validmind, checks dependencies then executes ONLY the Intro for Model Developers notebook with production heap tracking
33

4-
inputs: {}
4+
inputs:
5+
env_file:
6+
description: "Load the created .env file"
7+
required: true
58

69
runs:
710
using: "composite"
@@ -26,10 +29,21 @@ runs:
2629
pip install pycairo
2730
pip check
2831
32+
- name: Ensure .env file is available
33+
shell: bash
34+
id: find_env
35+
run: |
36+
if [ ! -f "${{ inputs.env_file }}" ]; then
37+
echo "Error: .env file not found at ${{ inputs.env_file }}"
38+
exit 1
39+
fi
40+
2941
- name: Execute ONLY the Intro for Model Developers notebook with heap production
3042
shell: bash
43+
if: ${{ steps.find_env.outcome == 'success' }}
3144
run: |
3245
cd site
46+
source ../${{ inputs.env_file }}
3347
quarto render --profile exe-prod notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb &> render_errors.log || {
3448
echo "Execute for intro_for_model_developers_EXECUTED.ipynb failed";
3549
cat render_errors.log;

‎.github/actions/staging-notebook/action.yml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
name: Execute staging notebook
22
description: Installs python3, validmind, checks dependencies then executes ONLY the Intro for Model Developers notebook with staging heap tracking
33

4-
inputs: {}
4+
inputs:
5+
env_file:
6+
description: "Load the created .env file"
7+
required: true
58

69
runs:
710
using: "composite"
@@ -26,10 +29,21 @@ runs:
2629
pip install pycairo
2730
pip check
2831
32+
- name: Ensure .env file is available
33+
shell: bash
34+
id: find_env
35+
run: |
36+
if [ ! -f "${{ inputs.env_file }}" ]; then
37+
echo "Error: .env file not found at ${{ inputs.env_file }}"
38+
exit 1
39+
fi
40+
2941
- name: Execute ONLY the Intro for Model Developers notebook with heap staging
3042
shell: bash
43+
if: ${{ steps.find_env.outcome == 'success' }}
3144
run: |
3245
cd site
46+
source ../${{ inputs.env_file }}
3347
quarto render --profile exe-staging notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb &> render_errors.log || {
3448
echo "Execute for intro_for_model_developers_EXECUTED.ipynb failed";
3549
cat render_errors.log;

‎.github/workflows/deploy-docs-prod.yaml‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -35,18 +35,31 @@ jobs:
3535
uses: dorny/paths-filter@v2
3636
id: filter
3737
with:
38+
base: prod
39+
head: staging
3840
filters: |
3941
notebooks:
4042
- 'site/notebooks/**'
4143
42-
# If yes then execute the prod notebook
43-
- name: Execute prod Intro for Model Developers notebook
44+
# If yes then create the .env file for use in execution step
45+
- name: Create .env file
4446
if: steps.filter.outputs.notebooks == 'true'
47+
id: create_env
48+
run: |
49+
touch .env
50+
echo VM_API_HOST=${{ secrets.PLATFORM_API_HOST }} >> .env
51+
echo VM_API_KEY=${{ secrets.PLATFORM_API_KEY }} >> .env
52+
echo VM_API_SECRET=${{ secrets.PLATFORM_API_SECRET }} >> .env
53+
echo VM_API_MODEL=${{ secrets.PLATFORM_DEV_MODEL }} >> .env
54+
cat .env
55+
56+
# Only execute the prod notebook if .env file is created
57+
- name: Execute prod Intro for Model Developers notebook
58+
if: ${{ steps.create_env.outcome == 'success' }}
4559
uses: ./.github/actions/prod-notebook
4660
id: execute-prod-notebook
47-
env:
48-
PLATFORM_API_KEY: ${{ secrets.PLATFORM_API_KEY }}
49-
PLATFORM_API_SECRET: ${{ secrets.PLATFORM_API_SECRET }}
61+
with:
62+
env_file: .env
5063

5164
# Prod bucket is in us-east-1
5265
- name: Configure AWS credentials

‎.github/workflows/deploy-docs-staging.yaml‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -35,18 +35,31 @@ jobs:
3535
uses: dorny/paths-filter@v2
3636
id: filter
3737
with:
38+
base: staging
39+
head: main
3840
filters: |
3941
notebooks:
4042
- 'site/notebooks/**'
4143
42-
# If yes then execute the staging notebook
43-
- name: Execute staging Intro for Model Developers notebook
44+
# If yes then create the .env file for use in execution step
45+
- name: Create .env file
4446
if: steps.filter.outputs.notebooks == 'true'
47+
id: create_env
48+
run: |
49+
touch .env
50+
echo VM_API_HOST=${{ secrets.PLATFORM_API_HOST }} >> .env
51+
echo VM_API_KEY=${{ secrets.PLATFORM_API_KEY }} >> .env
52+
echo VM_API_SECRET=${{ secrets.PLATFORM_API_SECRET }} >> .env
53+
echo VM_API_MODEL=${{ secrets.PLATFORM_DEV_MODEL }} >> .env
54+
cat .env
55+
56+
# Only execute the staging notebook if .env file is created
57+
- name: Execute staging Intro for Model Developers notebook
58+
if: ${{ steps.create_env.outcome == 'success' }}
4559
uses: ./.github/actions/staging-notebook
4660
id: execute-staging-notebook
47-
env:
48-
PLATFORM_API_KEY: ${{ secrets.PLATFORM_API_KEY }}
49-
PLATFORM_API_SECRET: ${{ secrets.PLATFORM_API_SECRET }}
61+
with:
62+
env_file: .env
5063

5164
# Staging bucket is in us-west-2
5265
- name: Configure AWS credentials

‎.github/workflows/validate-docs-site.yaml‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,18 +34,31 @@ jobs:
3434
uses: dorny/paths-filter@v2
3535
id: filter
3636
with:
37+
base: main
38+
head: ${{ github.head_ref }}
3739
filters: |
3840
notebooks:
3941
- 'site/notebooks/**'
4042
41-
# If yes then execute the demo notebook
42-
- name: Execute demo Intro for Model Developers notebook
43+
# If yes then create the .env file for use in execution step
44+
- name: Create .env file
4345
if: steps.filter.outputs.notebooks == 'true'
46+
id: create_env
47+
run: |
48+
touch .env
49+
echo VM_API_HOST=${{ secrets.PLATFORM_API_HOST }} >> .env
50+
echo VM_API_KEY=${{ secrets.PLATFORM_API_KEY }} >> .env
51+
echo VM_API_SECRET=${{ secrets.PLATFORM_API_SECRET }} >> .env
52+
echo VM_API_MODEL=${{ secrets.PLATFORM_DEV_MODEL }} >> .env
53+
cat .env
54+
55+
# Only execute the demo notebook if .env file is created
56+
- name: Execute demo Intro for Model Developers notebook
57+
if: ${{ steps.create_env.outcome == 'success' }}
4458
uses: ./.github/actions/demo-notebook
4559
id: execute-demo-notebook
46-
env:
47-
PLATFORM_API_KEY: ${{ secrets.PLATFORM_API_KEY }}
48-
PLATFORM_API_SECRET: ${{ secrets.PLATFORM_API_SECRET }}
60+
with:
61+
env_file: .env
4962

5063
- name: Test for warnings or errors
5164
run: |

‎site/Makefile‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,11 @@ DEST_DIR_NB := notebooks
44
DEST_DIR_PYTHON := _site/validmind
55
DEST_DIR_TESTS := tests
66
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD)
7+
PROFILE := exe-demo
8+
FILE_PATH := notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
79

810
# Define .PHONY target for help section
9-
.PHONY: help clean clone notebooks python-docs docs-site deploy-demo deploy-demo-branch delete-demo-branch deploy-prod deploy-staging release-notes
11+
.PHONY: help clean clone notebooks python-docs docs-site deploy-demo deploy-demo-branch delete-demo-branch deploy-prod deploy-staging release-notes execute
1012

1113
# Help section
1214
help:
@@ -24,6 +26,7 @@ help:
2426
@echo " deploy-staging Deploy docs staging site to s3://docs-ci-cd-staging/site/"
2527
@echo " help Display this help message (default target)"
2628
@echo " release-notes Generate release notes from pull requests since latest tag and update _quarto.yml"
29+
@echo " execute Execute a Jupyter Notebook (must have valid .env credentials set up in root)"
2730

2831
# Clean up source directory
2932
clean:
@@ -41,8 +44,8 @@ notebooks:
4144
@rm -f notebooks.zip
4245
@rm -rf $(DEST_DIR_NB)/ && mkdir -p $(DEST_DIR_NB)
4346
@cp -r $(SRC_DIR)/notebooks/. $(DEST_DIR_NB)/
44-
@echo "Checking out intro_for_model_developers_EXECUTED.ipynb from main ..."
45-
@git checkout main -- notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
47+
@echo "Duplicating notebooks/tutorials/intro_for_model_developers.ipynb for execution"
48+
@cp notebooks/tutorials/intro_for_model_developers.ipynb notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
4649
@echo "Copying LICENSE into notebooks ..."
4750
@cp -r $(SRC_DIR)/LICENSE $(DEST_DIR_NB)/
4851
@rm -rf $(DEST_DIR_NB)/code_sharing
@@ -77,8 +80,6 @@ get-source: clean clone notebooks python-docs test-descriptions
7780
# Get all source files
7881
docs-site: clean clone notebooks python-docs test-descriptions
7982
quarto render --profile development
80-
# quarto render notebooks/how_to/explore_tests.ipynb --execute
81-
# quarto render notebooks/how_to/explore_test_suites.ipynb --execute
8283

8384
# Deployment to https://docs-demo.vm.validmind.ai/
8485
deploy-demo:
@@ -120,3 +121,9 @@ deploy-staging:
120121
# Generate release notes
121122
release-notes:
122123
@python ../release-scripts/generate_release_objects.py
124+
125+
# Execute a Jupyter Notebook
126+
# Will default to `exe-demo` profile & the `notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb` if no input provided
127+
# To override: make execute PROFILE=select-profile FILE_PATH=notebooks/notebook-path-here.ipynb
128+
execute:
129+
quarto render --profile $(PROFILE) $(FILE_PATH)

‎site/developer/model-documentation/store-credentials-in-env-file.qmd‎

Lines changed: 58 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -31,26 +31,58 @@ Keeing model credentials in a separate file also allows for precise access contr
3131

3232
This is a hidden file, so you may need to change your settings to view it.
3333

34-
2. Locate the code snippet for your model.
34+
2. Locate the code snippet for your model:
3535

36-
- These credentials can be found on the **{{< fa rocket >}} Getting Started** page for models registered in the model inventory.
37-
- Copy the values from this page and paste them into your `.env` file in the following format:
36+
a. In the left sidebar, click **{{< fa cubes >}} Model Inventory**.
37+
b. Select a model by clicking on it or find your model by applying a filter or searching for it.[^3]
38+
c. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**.
39+
d. Locate the code snippet and click **{{< fa regular copy >}} Copy snippet to clipboard**.
40+
41+
3. Paste the credentials into your `.env` file in the following format:
3842

3943
```yaml
40-
VM_API_MODEL=<Model Identifier>
41-
VM_API_HOST=<API Host URL>
42-
VM_API_KEY=<API Key>s
43-
VM_API_SECRET=<API Secret>
44+
VM_API_HOST=<api_host>
45+
VM_API_KEY=<api_key>
46+
VM_API_SECRET=<api_secret>
47+
VM_API_MODEL=<model>
4448
```
4549

46-
3. Insert this code snippet above your model identifier credentials:
50+
::: {.column-margin}
51+
For example, if your credentials look like this:
52+
53+
```python
54+
import validmind as vm
55+
56+
vm.init(
57+
api_host = "https://api.prod.validmind.ai/api/v1/tracking",
58+
api_key = "key123",
59+
api_secret = "secret456",
60+
model = "model789"
61+
)
62+
```
63+
64+
Then your `.env` file should look like this:
65+
66+
```bash
67+
VM_API_HOST=https://api.prod.validmind.ai/api/v1/tracking
68+
VM_API_KEY=key123
69+
VM_API_SECRET=secret456
70+
VM_API_MODEL=model789
71+
```
72+
73+
:::
74+
75+
76+
4. Insert this code snippet above your model identifier credentials:
4777

4878
```python
4979
%load_ext dotenv
5080
%dotenv dev.env
5181
```
5282

53-
The updated notebook should look like this:
83+
5. Remove the inline credentials from `vm.init()`.[^4]
84+
85+
The updated code cell should now look like this:
5486

5587
```python
5688
%load_ext dotenv
@@ -59,19 +91,27 @@ Keeing model credentials in a separate file also allows for precise access contr
5991
import validmind as vm
6092
6193
vm.init(
62-
api_host = "http://localhost:3000/api/v1/tracking",
63-
model = "..."
6494
)
6595
```
6696

97+
::: {.column-margin}
98+
To enable monitoring when you are storing credentials in an `.env` file:^[[Enable monitoring](/guide/monitoring/enable-monitoring.qmd)]
6799

68-
4. Run the cell. Instead of using inline credentials, this cell will now load your model credentials from a `.env` file.
100+
```python
101+
%load_ext dotenv
102+
%dotenv .env
69103
70-
<!---
71-
## Troubleshooting
104+
import validmind as vm
105+
106+
vm.init(
107+
monitoring=True
108+
)
109+
```
110+
111+
:::
112+
113+
6. Run the cell. Instead of using inline credentials, this cell will now load your model credentials from a `.env` file.
72114

73-
[Include any common issues or errors that may arise during the task and how to resolve them.]
74-
--->
75115

76116
## What's next
77117

@@ -85,3 +125,5 @@ Keeing model credentials in a separate file also allows for precise access contr
85125

86126
[^2]: [Manage permissions](/guide/configuration/manage-permissions.qmd)
87127

128+
[^3]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models)
129+

‎site/guide/model-inventory/archive-delete-models.qmd‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Model deletion is also permanent and cannot be undone.
7070

7171
2. Select a model or find your model by applying a filter or searching for it.[^7]
7272

73-
3. First, archive your model.[^6]
73+
3. First, archive your model.[^8]
7474

7575
4. Then, under the [model stage]{.smallcaps} section, select **Deleted**.
7676

0 commit comments

Comments
 (0)