Skip to content

Docs: Work with metrics over time#449

Merged
validbeck merged 7 commits intomainfrom
beck/sc-6346/documentation-support-displaying-results
Sep 26, 2024
Merged

Docs: Work with metrics over time#449
validbeck merged 7 commits intomainfrom
beck/sc-6346/documentation-support-displaying-results

Conversation

@validbeck
Copy link
Copy Markdown
Collaborator

@validbeck validbeck commented Sep 25, 2024

Internal Notes for Reviewers

sc-6346

Work with metrics over time

Net-new doc: LIVE PREVIEW

Other updates

Screenshot 2024-09-25 at 11 12 24 AM

  • Added the "Work with metrics over time" page under Ongoing monitoring
  • The default permissions for the "Ongoing monitoring" sections were incorrect. They are developer-locked (I tested this as I ran both the notebooks for unit metrics & ongoing monitoring as a developer-only role, then tried to follow the instructions as a validator-only role).
  • Made sure that the other content referencing content blocks for test-driven etc. had ongoing monitoring included where relevant.
  • Cleaned up some of the awkward wording around instructions, especially in the "Enable monitoring" guide.

@validbeck validbeck added the internal Not to be externalized in the release notes label Sep 25, 2024
@validbeck validbeck self-assigned this Sep 25, 2024
@validbeck
Copy link
Copy Markdown
Collaborator Author

@gtagle @even-steven Can I get your accuracy check on this new doc? 🙏🏻

https://docs-demo.vm.validmind.ai/pr_previews/beck/sc-6346/documentation-support-displaying-results/guide/monitoring/work-with-metrics-over-time.html

If it looks good to you, Nik has requested that someone product/development side approve to reduce bottle-necking for PRs.

@github-actions
Copy link
Copy Markdown
Contributor

PR Summary

This pull request introduces several enhancements to the documentation related to monitoring and metrics over time in the ValidMind platform. The key changes include:

  1. New Documentation for Metrics Over Time: A new guide titled work-with-metrics-over-time.qmd has been added. This guide provides detailed instructions on how to view and add metrics over time to ongoing monitoring plans. It covers prerequisites, steps to add metrics, and how to view metadata associated with these metrics.

  2. Updates to Existing Documentation:

    • Model Documentation: Various updates have been made to the work-with-test-results.qmd, work-with-content-blocks.qmd, and _add-content-block.qmd files to include references to metrics over time and improve clarity.
    • Monitoring Guides: The enable-monitoring.qmd and review-monitoring-results.qmd files have been updated to reflect changes in the process of enabling and reviewing monitoring results.
    • Navigation and Structure: The _quarto.yml file has been updated to include the new metrics over time guide in the navigation structure.
  3. Minor Text and Formatting Adjustments: Several minor text changes and formatting adjustments have been made across various documentation files to improve readability and accuracy.

  4. New Images and GIFs: New images and GIFs have been added to illustrate the process of adding and viewing metrics over time.

Test Suggestions

  • Verify that the new 'Work with metrics over time' guide is accessible from the navigation menu and displays correctly.
  • Test the process of adding metrics over time as described in the new guide to ensure the instructions are accurate and complete.
  • Check all updated documentation files for broken links or missing images.
  • Ensure that the new images and GIFs are displayed correctly and enhance the understanding of the documentation.
  • Review the changes in the _quarto.yml file to confirm that the navigation structure is updated correctly.

@github-actions
Copy link
Copy Markdown
Contributor

PR Summary

This pull request introduces several enhancements to the monitoring documentation and the handling of metrics over time within the ValidMind platform. The key changes include:

  1. New Documentation for Metrics Over Time:

    • Added a new guide work-with-metrics-over-time.qmd that provides detailed instructions on how to add and view metrics over time in the ongoing monitoring plans.
    • This guide includes prerequisites, steps to add metrics, and how to view metadata associated with these metrics.
  2. Updates to Existing Documentation:

    • Updated various documentation files to include references to the new metrics over time feature.
    • Enhanced descriptions and instructions for adding content blocks, including the new metric over time blocks.
    • Improved clarity and consistency in the instructions for working with test results and ongoing monitoring.
  3. Configuration Changes:

    • Updated _quarto.yml to include the new metrics over time guide in the monitoring section.
    • Modified ongoing-monitoring.qmd to include the new guide in the listing of ongoing monitoring resources.
  4. Minor Fixes and Improvements:

    • Corrected some terminology and improved the clarity of instructions in various documentation files.
    • Updated image alt texts and descriptions for better accessibility and understanding.

These changes aim to enhance the user experience by providing clearer and more comprehensive documentation for monitoring and metrics over time features.

Test Suggestions

  • Verify that the new 'Work with metrics over time' guide is accessible from the monitoring section.
  • Check that all links and references to the new metrics over time feature are working correctly.
  • Ensure that the instructions for adding and viewing metrics over time are clear and accurate.
  • Test the process of adding a metric over time block to the documentation and verify that it functions as described.
  • Review the updated image alt texts and descriptions for accuracy and clarity.

@github-actions
Copy link
Copy Markdown
Contributor

PR Summary

This pull request introduces several enhancements to the monitoring documentation and the addition of a new feature for working with metrics over time. The key changes include:

  1. New Documentation for Metrics Over Time:

    • Added a new guide work-with-metrics-over-time.qmd that explains how to view and add metrics over time to ongoing monitoring plans.
    • This guide includes steps for adding metrics, viewing metadata, and understanding the importance of tracking model performance over time.
  2. Updates to Existing Documentation:

    • Updated various documentation files to include references to the new metrics over time feature.
    • Improved clarity and consistency in the instructions for adding and managing content blocks, including test-driven and metric over time blocks.
    • Enhanced the descriptions and instructions for enabling ongoing monitoring and reviewing monitoring results.
  3. UI and Content Adjustments:

    • Adjusted the UI elements and instructions to reflect the new options available for metrics over time.
    • Updated screenshots and GIFs to showcase the new features and improved user interface.
  4. Minor Corrections and Improvements:

    • Fixed typos and improved the readability of the documentation.
    • Ensured that all references and links are up-to-date and accurate.

These changes aim to provide better guidance for users on how to effectively monitor their models and utilize the new metrics over time feature.

Test Suggestions

  • Verify that the new 'Work with metrics over time' guide is accessible and renders correctly.
  • Test the process of adding a metric over time block to model documentation and ongoing monitoring plans.
  • Ensure that the updated instructions for enabling ongoing monitoring are clear and functional.
  • Check that all links and references in the documentation are correct and lead to the appropriate sections.
  • Validate that the UI elements and screenshots match the described steps in the documentation.

@validbeck
Copy link
Copy Markdown
Collaborator Author

@johnwalz97 Can you accuracy check this new doc for the explanation as to why metrics over time are relevant/used?

https://docs-demo.vm.validmind.ai/pr_previews/beck/sc-6346/documentation-support-displaying-results/guide/monitoring/work-with-metrics-over-time.html

@github-actions
Copy link
Copy Markdown
Contributor

PR Summary

This pull request introduces several enhancements to the monitoring documentation and the addition of a new feature for working with metrics over time. The key changes include:

  1. New Documentation for Metrics Over Time:

    • Added a new guide work-with-metrics-over-time.qmd that explains how to view and add metrics over time to ongoing monitoring plans.
    • This guide includes steps for adding metrics, viewing metadata, and ensuring the accuracy of the data.
  2. Updates to Existing Documentation:

    • Updated various documentation files to include references to the new metrics over time feature.
    • Improved clarity and consistency in the instructions for adding and managing content blocks, test results, and ongoing monitoring.
    • Enhanced descriptions and instructions for using the ValidMind platform, including better explanations for roles and permissions.
  3. UI Enhancements:

    • Updated the _quarto.yml configuration to include the new metrics over time guide in the monitoring section.
    • Improved the layout and navigation within the monitoring documentation by adding new entries and updating existing ones.
  4. Minor Fixes and Improvements:

    • Corrected typos and improved the readability of several documentation files.
    • Updated image alt texts and descriptions for better accessibility and clarity.

These changes aim to provide a more comprehensive and user-friendly experience for developers and validators working with the ValidMind platform's monitoring features.

Test Suggestions

  • Verify that the new 'Metrics over time' guide is accessible from the monitoring section in the documentation.
  • Test the steps outlined in the 'work-with-metrics-over-time.qmd' guide to ensure they are accurate and functional.
  • Check that all links and references to the new metrics over time feature are working correctly.
  • Ensure that the updated instructions for adding content blocks, test results, and ongoing monitoring are clear and accurate.
  • Validate that the UI changes in _quarto.yml correctly reflect the new documentation structure.

Copy link
Copy Markdown
Contributor

@johnwalz97 johnwalz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick nitpick typo (sorry lol)... but other than that the content looks great to me 👍

Copy link
Copy Markdown

@even-steven even-steven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍🏽

@github-actions
Copy link
Copy Markdown
Contributor

A PR preview is available: Preview URL

@validbeck validbeck merged commit 820b91e into main Sep 26, 2024
@validbeck validbeck deleted the beck/sc-6346/documentation-support-displaying-results branch September 26, 2024 17:07
nrichers added a commit that referenced this pull request Oct 3, 2024
* Fix staging deploy in Makefile (#428)

* Simplify merge strategy for staging (#373)

* Simplify merge strategy

* Minor workflow tweaks

* Switch to test branches

* Add comment to trigger workflow

* Add comment to trigger workflow

* Add comment to trigger workflow

* Switch to squash merge

* Add comment to trigger workflow

* Remove comment to trigger workflow

* Remove comment to trigger workflow

* Remove comment to trigger workflow

* Remove comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Try --ff strategy

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Revert to --no-ff strategy

* Try reset --hard

* Switch to --merge for PRs

* Switch back to merge --no-ff

* Try merge --ff again

* Add test comment to trigger workflow

* Switch back to merge --no-ff

* Add test comment to trigger workflow

* Switch back to squash merging for PRs

* Add test comment to trigger workflow

* Remove test comments

* Revert to --merge for PRs

* Testing PR#373 - push to branch from local

* Add comment to trigger workflow

* Add comment to trigger workflow

* Add comment to trigger workflow

* Switch to squash merge

* Add comment to trigger workflow

* Remove comment to trigger workflow

* Remove comment to trigger workflow

* Remove comment to trigger workflow

* Remove comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Switch to --merge for PRs

* Add test comment to trigger workflow

* Add test comment to trigger workflow

* Switch back to squash merging for PRs

* Add test comment to trigger workflow

* Remove test comments

* Revert to --merge for PRs

* Undo test changes

* Undo test comment

---------

Co-authored-by: Beck <[email protected]>

* I'm dumb (#431)

* Fix branch in merge-main-into-staging.yaml (#432)

* Styled screenshots & cleaned up current screenshots (#420)

* Added a .screenshot CSS class

* Sample image

* Changed class for About -releases

* Changed class for Get Started & edited images to be uniform

* Changed class for guides/configuration

* Changed class for guides/model-workflows

* Changed class for guides/model-inventory

* Changed class for guides/model-documentation

* Changed class for guides/model-validation

* Changed class for guides/monitoring

* Changed class for developer/

* Changed class for releases/ 1st pass

* Cropped images for 2024-may-22 -1 gif

* Reverting changes for a gif to crop again

* Cropped images for releases 2nd pass

* Cropped images for releases 3rd  pass

* Cropped images for releases 4th pass

* More tweaks

* Videos styling

* Tweaks to JH quickstart

* More cropping

* Applied class to training

* Annotated screenshots in guides/

* Currency field type (#423)

* Preview of Generate with AI docs sections (#419)

* Preview of Generate with AI docs sections

* Release notes draft

* Image edits

* Editing release

* Release notes done

* Final tweaks + new gif

* Oops I lied

* anchor links

* Added attachment field type (#442)

* Added attachment field type

* Wording tweak

* Missed one

* Pulling in latest from developer-framework PR#195 (#438)

* Pulling in latest from developer-framework

* Added .screenshot class to model inventory fields

* Added aliases for index.html & training-overview.html (#444)

* Docs: Work with metrics over time (#449)

* WIP

* Key concepts draft

* Clarified  monitoring instructions now w/ accurate default permissions

* Final draft of Work with metrics over time

* Adjusted Work with content blocks

* Fixing display issue for test description listings on monitoring

* Update site/guide/monitoring/work-with-metrics-over-time.qmd

Co-authored-by: John Halz <[email protected]>

---------

Co-authored-by: John Halz <[email protected]>

* Templates for videos (#415)

* Add docs.validmind.ai/training redirect

* Update backgrounds, remove slide numbering, add slide links

* Add initial presentation with sample intro slides

* Add slides

* Update video slides

* Hide controls

* Organize slides into sections by type

* Minor update for adding tests

* Calculation field instructions & example (#443)

* Calculation field instructions & example

* Tweak

* Moved calculation screenshot into its own line

* Added Juan's magic formula

* Set up backfilled redirects for renamed pages (#454)

* 1st pass of links that used to be on developer-framework

* 1st pass of other external links to docs

* 1st pass of newer renames

* Big restructure of guides alias links 1st pass

* Restructure of Documenting models 1st pass

* 2nd pass of big restructure

* Final pass of big restructure

* Added alias redirects to style guide

* Embed Validating Models 101 playlist into Guides section (#446)

* Embed Validating Models 101 playlist into Guides section

* Remove whitespace

* Update site/guide/guides.qmd

Co-authored-by: Beck <[email protected]>

* Update site/guide/guides.qmd

Co-authored-by: Beck <[email protected]>

* Undo lead-in text for video

---------

Co-authored-by: Beck <[email protected]>

* Embed "How do I ... add tests?" video in developer docs (#450)

* Add how do I add test video

* Switch to iframe for video embed

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Beck <[email protected]>
Co-authored-by: John Halz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants