Conversation
juanmleng
left a comment
There was a problem hiding this comment.
Looks great. Just a suggestion to perhaps expand the review coverage to assumptions, expert judgment, and reporting:
Model development: Focuses on the model's development process, including training, evaluation, explainability, interpretability, and diagnosis. Highlights the model’s performance, identifies strengths and weaknesses, and ensures that any potential issues such as overfitting are addressed. Evaluates the assumptions made and examines the qualitative information and judgments to ensure they are conducted appropriately and systematically.
Monitoring and governance: Evaluates the ongoing strategies for monitoring the model's performance and ensuring compliance with regulatory and ethical standards. Involves checking the implementation of the monitoring plan and governance strategies to maintain the model's efficacy over time. Covers reporting outputs to ensure transparency and accuracy in the model's documented results.
Thank you, @juanmleng! I've made these updates verbatim as suggested. |
sc-4973/better-guidance-around-what-review-entails
Internal Notes for Reviewers
This PR makes an incremental improvement to our validator training and related documentation by being more explicit about what the concept of 'review' entails:
Information here is derived from a combination of how our template structures model documentation by default, chapter 4 of Joshi's "Managing Risk of Financial Models", and a convo with ChatGPT to investigate our current user guide content.
Output
Main glossary
New glossary entry:
"Review model documentation" topic
Validator training
External Release Notes
We clarified what the concept of review entails by adding a glossary entry, including that in a new key concepts section in our topic for reviewing model documentation, and updating our validator training to be more explicit. [../../training/training-for-model-validators.html]