PR Summary

This pull request introduces enhancements to the documentation and functionality related to LLM-generated test descriptions within the ValidMind platform. The key changes include:

1. Documentation Updates:

   • Added a new notebook add_context_to_llm_descriptions.ipynb that provides a comprehensive guide on how to add custom use case and test-specific context to LLM-generated test descriptions. This notebook includes detailed steps and examples for setting up the environment, initializing datasets and models, and configuring context for various tests.
   • Updated the testing-overview.qmd, faq-testing.qmd, and work-with-content-blocks.qmd files to include references and explanations about the new capability to add context to LLM-generated test descriptions.
   • Modified the release notes to reflect the addition of the new notebook and its purpose.

2. Functional Enhancements:

   • Introduced the ability to add custom context to LLM-generated test descriptions, allowing users to provide additional information about the test or use case, which can be crucial for aligning with organizational policies and specific use cases.
   • The notebook demonstrates how to enable and disable this feature, as well as how to set both use case and test-specific contexts.

These changes aim to improve the flexibility and relevance of LLM-generated test descriptions, making them more aligned with user-specific requirements and organizational standards.

Test Suggestions

• Verify that the new notebook add_context_to_llm_descriptions.ipynb runs without errors and produces the expected outputs.
• Test the functionality of adding custom context to LLM-generated test descriptions by setting different contexts and ensuring the descriptions reflect these changes.
• Check the documentation updates in testing-overview.qmd, faq-testing.qmd, and work-with-content-blocks.qmd for accuracy and clarity.
• Ensure that the release notes correctly describe the new features and provide accurate links to the new notebook.

@juanmleng Can you review the above additions for accuracy? 🙏🏻

@kristof87 & @mehdi0501 Any other messaging you'd like/thoughts on the delivery so far? Thank you!

PR Summary

This pull request introduces enhancements to the documentation and functionality of the ValidMind platform, specifically focusing on the ability to add custom context to LLM-generated test descriptions. The changes include:

1. Documentation Updates:

   • Added a new notebook add_context_to_llm_descriptions.ipynb that provides a comprehensive guide on how to add custom use case and test-specific context to LLM-generated test descriptions. This notebook includes detailed steps and examples for setting up the environment, initializing datasets and models, and configuring context for various tests.
   • Updated the testing-overview.qmd, faq-testing.qmd, and work-with-content-blocks.qmd files to include references and explanations about the new feature of adding context to LLM-generated descriptions.
   • Modified the release notes to reflect the addition of the new notebook and its purpose.

2. Functional Enhancements:

   • Introduced the ability to enable or disable custom context for LLM-generated descriptions through environment variables. This allows users to provide additional information about the test or use case, which can be used to tailor the generated descriptions to better fit organizational policies and specific use cases.
   • Provided examples of how to set test-specific context for various tests such as DatasetDescription, ClassImbalance, HighCardinality, and others, demonstrating how to incorporate business-specific criteria and thresholds into the test descriptions.

Test Suggestions

• Verify that the new notebook add_context_to_llm_descriptions.ipynb runs without errors and produces the expected outputs.
• Test the functionality of enabling and disabling custom context for LLM-generated descriptions using the environment variable VALIDMIND_LLM_DESCRIPTIONS_CONTEXT_ENABLED.
• Check that the context provided in the environment variable VALIDMIND_LLM_DESCRIPTIONS_CONTEXT is correctly incorporated into the LLM-generated descriptions.
• Ensure that the documentation updates in testing-overview.qmd, faq-testing.qmd, and work-with-content-blocks.qmd are accurate and provide clear guidance on using the new feature.
• Validate that the links and references in the release notes are correct and lead to the appropriate resources.

PR Summary

This pull request introduces enhancements to the documentation and functionality of the ValidMind platform, focusing on the ability to add custom context to LLM-generated test descriptions. The changes include:

1. Documentation Updates:

   • Added a new notebook add_context_to_llm_descriptions.ipynb that provides a comprehensive guide on how to add custom use case and test-specific context to LLM-generated test descriptions. This notebook includes detailed steps and examples for setting up the environment, initializing datasets and models, and configuring context for various tests.
   • Updated the testing-overview.qmd, faq-testing.qmd, and work-with-content-blocks.qmd files to include references and explanations about the new feature of adding context to LLM-generated descriptions.
   • Modified the release notes to reflect the addition of the new notebook and its purpose.

2. Functional Enhancements:

   • Introduced the ability to enable or disable custom context for LLM-generated descriptions through environment variables.
   • Provided examples of how to set use case context and test-specific context for different types of tests, such as DatasetDescription, ClassImbalance, HighCardinality, and others.

These changes aim to improve the flexibility and relevance of test descriptions generated by the LLM, allowing users to tailor the output to better fit their specific use cases and organizational requirements.

Test Suggestions

• Test the new notebook add_context_to_llm_descriptions.ipynb to ensure it runs without errors and produces the expected outputs.
• Verify that enabling and disabling the custom context through environment variables works as intended.
• Check that the LLM-generated descriptions reflect the custom context provided in various test scenarios.
• Ensure that the documentation updates are correctly linked and provide accurate information about the new feature.
• Test the integration of the new feature with existing tests to ensure backward compatibility.