Internal Notes for Reviewers

sc-7406

ValidMind Library updates

This PR pulls in the latest changes from updated validmind-library via make get-source. I can confirm that the edits show up in the rendered files.

For example: LIVE API REFERENCE PREVIEW

Live execution of intro_for_model_developers_EXECUTED.ipynb

I've used Quarto's native JupyterLab functionality to render this notebook on the fly, and it only cost me some of my sanity — what a bargin.

• Now, the executed notebook RENDERS THE MISSING WIDGET TEMPLATES which are now fully interactive!!!
• It is also now set to page-layout: full to cover more of the negative-space left by the missing sidenav.

Old	New
OLD VERSION	NEW LIVE PREVIEW

How does this work? Well:

Global Jupyter Notebook execution settings

In _quarto.yml, there are now some global execute settings determining behaviour for what Quarto should do when it renders .ipynb files:

• Execution for notebooks is disabled by default
• Notebooks shouldn't be re-executed during re-renders

intro_for_model_developers_EXECUTED.ipynb front matter

There is now a raw cell at the top of this version of the notebook that determines the granular execution behaviour for rendering this file:

• This front matter is hidden on the rendered version
• Execution for notebooks is enabled
• Notebook should be re-executed on re-renders

Local output for executed notebook added to .gitignore

If you preview or render the files locally, this notebook will execute and we don't want the output cruft (since the workflows take care of the execution for our hosted sites):

• Quarto rendered version of the execution ignored: site/_freeze/
• Actual output of that particular notebook ignored: site/notebooks/tutorials/my_tests/

Secure api_key & api_secret

New "Repository secrets"

• For use in the updated GitHub workflows, two (2) new secrets have been added:

1. PLATFORM_API_KEY
2. PLATFORM_API_SECRET

• These secrets are linked to this ValidMind Academy model: cm3j9lu5208mf21ifij2tp6bk
  
• Obviously, the code in the notebook references both the GH workflow secrets and your local .env secrets so that it works in both environments — see below.

.env.example

• Locally, I also have the secrets added to my .env file. WHEN YOU RUN THE NEXT PREVIEW OR RENDER LOCALLY, you'll need to replace this with the Academy org's model's key and secret so that the notebook executes properly, as well as add yourself as a developer to the model, etc.
• I've updated the .env.example as well:

Updated GitHub workflows

Some new jobs have been added to all three of our "render docs" workflows:

• Install python3 for Jupyter Notebooks: This installs the version of python3 required for ValidMind & upgrades pip for use in Check dependencies.
• Install validmind for notebook execution: This installs all the validmind packages and dependencies required to run the "Intro to Model Developers" notebook with minimal warnings and errors. (There are a few weird ones, I went on quite a journey.)
• Check dependencies: This installs pycairo required for check and then runs the dependency check.

The following job has been updated to include the repository env secrets:

• Render docs site:

n.b.

This PR won't be merged until #529 is approved and merged as they are intertwined.

Local execution of intro_for_model_developers_EXECUTED.ipynb

EDIT: We discussed this and we will add this officially to the .gitignore in a follow-up PR so we can just leave this file untouched and not override it accidentally with local changes:

• You'll either have to add the Academy model's valid key and secrets as the model's developer so that the notebook renders/previews fine locally (as long as you have most of the packages/dependencies sorted..... which they might not be (WARNING: 🐰 🕳️ ))
• OR, comment out the following in the front matter and temporarily add this file to your .gitignore so that the changes don't get sent up to remote:

    code-tools: true
    include-after-body: 
      - https://unpkg.com/@jupyter-widgets/html-manager@*/dist/embed-amd.js
execute: 
  enabled: true
  freeze: auto

SHAPGlobalImportance.py error

• This test is returning some errors but the rest of the tests are still being logged to the platform.
• Same error under two sections:

1. "Run the model evaluation tests"
2. "Viewing and updating the configuration for the entire model documentation template > Update the config"

ERROR(validmind.vm_models.test_suite.test): Failed to run test 'validmind.model_validation.sklearn.SHAPGlobalImportance': (TypeError) loop of ufunc does not support argument 0 of type float which has no callable rint method

• This didn't happen the last time I executed the notebook and I spent about half a day fiddling with dependencies and script edits to no avail, so I've just logged a Shortcut Story on advice of Andres: sc-7456
• The good thing is, since the notebook is being executed on the fly, in theory if it is an issue with the Library or the test, once those updates are pulled in the errors in the rendered notebook should resolve (I think).

Developer Fundamentals iframe

• This is still linking to the TOP-LEVEL docs-demo rendered version, because for some reason the rendered relative links in the iframe are finicky. This feels similar to the issue with the original relative link buttons in the release previews for the python docs.
• BUT, since the executed/rendered version is fine outside of that relative link in the preview, it should likely be fine once we send the new executed notebook up to staging and prod.
• You can inspect this yourself by going into the training, and then on the first slide that has the relative executed notebook as the background, click View Frame Source:
  

Added render run-time

Yeah, I know. I'm sorry. For local versions, you can use the .gitignore hack for the specific notebook to get around it. I think this is pretty cool functional trade-off though.

Edit: Variables are messed up on mobile view & breadcrumbs

This applies to titles and sidebar links in _quarto.yml:

Applied a workaround for now (just using the plain-text term).