Skip to content

Loading a docarray from a database which is setup with subindices does not work #555

@samsja

Description

@samsja

context

Loading a docarray from a database which is setup with subindices does not work #555

example :

from docarray import Document, DocumentArray, dataclass
from docarray.typing import Image, Text

@dataclass
class MyDocument:
    image: Image
    paragraph: Text

# extend with Documents, including embeddings
_docs = [(MyDocument( image='https://docarray.jina.ai/_images/apple.png', paragraph='hello world')),
         (MyDocument( image='https://pocket-syndicated-images.s3.amazonaws.com/6205832f02f07.jpg', paragraph='bread')),
         (MyDocument( image='https://cdn.arstechnica.net/wp-content/uploads/2022/09/stable_diffusion_on_lexica-800x448.jpg', paragraph='ai is going to eat you'))]

da = DocumentArray(
    storage='sqlite',  # use SQLite as vector database
    config={'connection': 'jina4.db', 'table_name': 'test4'},
    subindex_configs={'@.[image]': {'connection': 'jina4.db', 'table_name': 'test5'}, '@.[paragraph]': {'connection': 'jina4.db', 'table_name': 'test6'}},  # set up subindices for image and description
)
da.summary()

for item in _docs:
  d = Document(item)
  da.append(d)

da = DocumentArray(
    storage='sqlite',  # use SQLite as vector database
    config={'connection': 'jina4.db', 'table_name': 'test4'},
    subindex_configs={'@.[image]': {'connection': 'jina4.db', 'table_name': 'test5'}, '@.[paragraph]': {'connection': 'jina4.db', 'table_name': 'test6'}},  # set up subindices for image and description
)
da.summary()
sqlite3.IntegrityError: UNIQUE constraint failed: test5.doc_id

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions