fix: calculate relevant docs on index instead of queries#950
Merged
samsja merged 5 commits intodocarray:mainfrom Dec 16, 2022
Merged
Conversation
Signed-off-by: Michael Guenther <[email protected]>
JoanFM
requested changes
Dec 15, 2022
Member
JoanFM
left a comment
There was a problem hiding this comment.
Put it as draft until really ready for review
| match_batch_size=1, | ||
| limit=10, | ||
| ) | ||
| print(res) |
Member
There was a problem hiding this comment.
remove print, there should be assertions
Signed-off-by: Michael Guenther <[email protected]>
LMMilliken
reviewed
Dec 15, 2022
Comment on lines
+656
to
+657
| def test_embed_and_evaluate_on_real_data(two_embed_funcs, kwargs): | ||
| metric_names = ['precision_at_k', 'reciprocal_rank'] | ||
| metric_names = ['precision_at_k', 'reciprocal_rank', 'recall_at_k'] |
There was a problem hiding this comment.
Maybe add a test case for exclude_self = True/False
Codecov ReportBase: 85.21% // Head: 77.21% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## main #950 +/- ##
==========================================
- Coverage 85.21% 77.21% -8.00%
==========================================
Files 155 155
Lines 8048 8049 +1
==========================================
- Hits 6858 6215 -643
- Misses 1190 1834 +644
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
Signed-off-by: Michael Guenther <[email protected]>
Signed-off-by: Michael Guenther <[email protected]>
a94f76d to
e1a74b9
Compare
JoanFM
approved these changes
Dec 15, 2022
gmastrapas
approved these changes
Dec 16, 2022
Signed-off-by: Michael Guenther <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goals:
The
num_relevant_documents_per_labelis calculated in theembed_and_evaluatemethod on the documents inself. However, this is only correct ifselfis matched against itself. Instead it should be calculated on theindex_dataattribute if it is provided.check and update documentation, if required. See guide