Skip to content

feat(redis): implement Redis storage backend and unit tests#452

Merged
hanxiao merged 107 commits intodocarray:mainfrom
AnneYang720:feat-redis-backend
Aug 26, 2022
Merged

feat(redis): implement Redis storage backend and unit tests#452
hanxiao merged 107 commits intodocarray:mainfrom
AnneYang720:feat-redis-backend

Conversation

@AnneYang720
Copy link
Copy Markdown
Contributor

@AnneYang720 AnneYang720 commented Jul 25, 2022

This PR will implement Redis storage backend and its unit test according to #445

Copy link
Copy Markdown

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments

Copy link
Copy Markdown
Member

@JoanFM JoanFM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these unit tests are excessive.You are only testing private methods. In general, we should test the public interface. These tests may make changing underlying implementation very hard

@bwanglzu
Copy link
Copy Markdown

bwanglzu commented Jul 27, 2022

please also fix the commit messages, see error in commit-lint, use git amend

@codecov
Copy link
Copy Markdown

codecov bot commented Jul 27, 2022

Codecov Report

Merging #452 (5f94757) into main (7c91c7b) will increase coverage by 0.32%.
The diff coverage is 94.13%.

@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
+ Coverage   86.53%   86.86%   +0.32%     
==========================================
  Files         134      140       +6     
  Lines        6704     6994     +290     
==========================================
+ Hits         5801     6075     +274     
- Misses        903      919      +16     
Flag Coverage Δ
docarray 86.86% <94.13%> (+0.32%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
docarray/array/document.py 70.76% <62.50%> (-1.17%) ⬇️
docarray/array/storage/redis/seqlike.py 86.66% <86.66%> (ø)
docarray/array/storage/redis/find.py 94.36% <94.36%> (ø)
docarray/array/storage/redis/backend.py 96.19% <96.19%> (ø)
docarray/array/storage/redis/getsetdel.py 96.77% <96.77%> (ø)
docarray/array/redis.py 100.00% <100.00%> (ø)
docarray/array/storage/redis/__init__.py 100.00% <100.00%> (ø)
docarray/array/storage/base/helper.py 92.30% <0.00%> (+2.56%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@bwanglzu bwanglzu marked this pull request as ready for review August 1, 2022 09:02
@bwanglzu bwanglzu linked an issue Aug 1, 2022 that may be closed by this pull request
@bwanglzu bwanglzu mentioned this pull request Aug 1, 2022
7 tasks
Copy link
Copy Markdown
Member

@JoanFM JoanFM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of all these tests for now I would keep only test_init_storage

@bwanglzu bwanglzu marked this pull request as draft August 1, 2022 11:56
@bwanglzu
Copy link
Copy Markdown

bwanglzu commented Aug 1, 2022

@AnneYang720 let's commit everything on this branch without creating seperate PRs, we'll make it ready-to-review until everything is here

Copy link
Copy Markdown
Member

@alaeddine-13 alaeddine-13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the documentation, LGTM, well done !


### Vector search with filter query

One can perform Vector Similarity Search based on FLAT or HNSW algorithm and pre-filter results using a filter query that is based on [MongoDB's Query](https://www.mongodb.com/docs/manual/reference/operator/query/). We currently support a subset of those selectors:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wouldn't mention MongoDB here. BTW we can refer it as domain specific language in the future or DSL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's fine we already mention it here: https://docarray.jina.ai/fundamentals/documentarray/find/

Copy link
Copy Markdown

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work on the docs! left some minor comments


### Vector search with filter query

One can perform Vector Similarity Search based on FLAT or HNSW algorithm and pre-filter results using a filter query that is based on [MongoDB's Query](https://www.mongodb.com/docs/manual/reference/operator/query/). We currently support a subset of those selectors:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's fine we already mention it here: https://docarray.jina.ai/fundamentals/documentarray/find/

Copy link
Copy Markdown
Member

@JohannesMessner JohannesMessner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some rephrasings

@hanxiao hanxiao merged commit c2235de into docarray:main Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Redis storage backend - backend.py

6 participants