Skip to content

fix: fix load from a database with subindices#581

Merged
JoanFM merged 1 commit intomainfrom
fix-datbase-load-subindices
Oct 3, 2022
Merged

fix: fix load from a database with subindices#581
JoanFM merged 1 commit intomainfrom
fix-datbase-load-subindices

Conversation

@dongxiang123
Copy link
Copy Markdown
Contributor

@dongxiang123 dongxiang123 commented Sep 30, 2022

ATM we could not read from a database which has been setup with sub-indices. This pr fix this (original issue #555 )

from docarray import Document, DocumentArray, dataclass
from docarray.typing import Image, Text

@dataclass
class MyDocument:
    image: Image
    paragraph: Text

# extend with Documents, including embeddings
_docs = [(MyDocument( image='https://docarray.jina.ai/_images/apple.png', paragraph='hello world'))]

da = DocumentArray(
    storage='sqlite',  # use SQLite as vector database
    config={'connection': 'jina4.db', 'table_name': 'test4'},
    subindex_configs={'@.[image]': {'connection': 'jina4.db', 'table_name': 'test5'}, '@.[paragraph]': {'connection': 'jina4.db', 'table_name': 'test6'}},  # set up subindices for image and description
)
da.summary()

for item in _docs:
  d = Document(item)
  da.append(d)

da = DocumentArray(
    storage='sqlite',  # use SQLite as vector database
    config={'connection': 'jina4.db', 'table_name': 'test4'},
    subindex_configs={'@.[image]': {'connection': 'jina4.db', 'table_name': 'test5'}, '@.[paragraph]': {'connection': 'jina4.db', 'table_name': 'test6'}},  # set up subindices for image and description
)
da.summary()

This will now work

@JoanFM
Copy link
Copy Markdown
Member

JoanFM commented Sep 30, 2022

Make sure to fix the commit lint and to format the python code in the description

@JoanFM
Copy link
Copy Markdown
Member

JoanFM commented Sep 30, 2022

Also dataarray is nothing in docarray, just use fix

@dongxiang123 dongxiang123 force-pushed the fix-datbase-load-subindices branch from ef3746d to c5b15fe Compare September 30, 2022 09:00
@dongxiang123 dongxiang123 changed the title fix(dataarray): Fix load from a database with subindices fix: fix load from a database with subindices Sep 30, 2022
@codecov
Copy link
Copy Markdown

codecov bot commented Sep 30, 2022

Codecov Report

Merging #581 (0e6d96f) into main (57073c0) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #581      +/-   ##
==========================================
+ Coverage   86.40%   86.43%   +0.03%     
==========================================
  Files         134      134              
  Lines        6714     6716       +2     
==========================================
+ Hits         5801     5805       +4     
+ Misses        913      911       -2     
Flag Coverage Δ
docarray 86.43% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
docarray/array/storage/base/backend.py 87.75% <100.00%> (+0.52%) ⬆️
docarray/array/mixins/plot.py 67.96% <0.00%> (+0.86%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@dongxiang123 dongxiang123 force-pushed the fix-datbase-load-subindices branch from c5b15fe to c45014e Compare September 30, 2022 09:59
@dongxiang123 dongxiang123 force-pushed the fix-datbase-load-subindices branch from c45014e to fef00bd Compare September 30, 2022 12:31
@github-actions github-actions bot added size/m and removed size/s labels Sep 30, 2022
@dongxiang123 dongxiang123 force-pushed the fix-datbase-load-subindices branch 2 times, most recently from 7db7e8f to 6446dee Compare October 1, 2022 08:14
@github-actions github-actions bot added size/s and removed size/m labels Oct 1, 2022
@dongxiang123 dongxiang123 force-pushed the fix-datbase-load-subindices branch from 6446dee to 0e6d96f Compare October 1, 2022 08:45
@JoanFM JoanFM merged commit cf98028 into main Oct 3, 2022
@JoanFM JoanFM deleted the fix-datbase-load-subindices branch October 3, 2022 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Loading a docarray from a database which is setup with subindices does not work

2 participants