Skip to content

ModuleNotFoundError when using elasticsearch as storage backend #529

@violenil

Description

@violenil

Code examples from this documentation with elasticsearch as storage result in ModuleNotFoundError.

Minimal example:

from docarray import DocumentArray, Document

da = DocumentArray(
    storage='elasticsearch',
    config={'index_name': 'new_index', 'n_dim': 128},
)

with da:
    da.extend([Document() for _ in range(1000)])

da.summary()

Example code runs through and displays .summary():

╭─────────────── Documents Summary ───────────────╮
│                                                 │
│   Type                   DocumentArrayElastic   │
│   Length                 1000                   │
│   Homogenous Documents   True                   │
│   Common Attributes      ('id',)                │
│   Multimodal dataclass   False                  │
│                                                 │
╰─────────────────────────────────────────────────╯
╭───────────────────── Attributes Summary ─────────────────────╮
│                                                              │
│   Attribute   Data type   #Unique values   Has empty value   │
│  ──────────────────────────────────────────────────────────  │
│   id          ('str',)    1000             False             │
│                                                              │
╰──────────────────────────────────────────────────────────────╯
╭──────── DocumentArrayElastic Config ────────╮
│                                             │
│   n_dim             128                     │
│   distance          cosine                  │
│   hosts             http://localhost:9200   │
│   index_name        new_index             │
│   es_config         {}                      │
│   index_text        False                   │
│   tag_indices       []                      │
│   batch_size        64                      │
│   ef_construction   None                    │
│   m                 None                    │
│   columns           []                      │
│                                             │
╰─────────────────────────────────────────────╯

But then raises ModuleNotFoundError:

Exception ignored in: <function BaseGetSetDelMixin.__del__ at 0x7fa010267e50>
Traceback (most recent call last):
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/docarray/array/storage/base/getsetdel.py", line 330, in __del__
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/docarray/array/storage/elastic/getsetdel.py", line 128, in _save_offset2ids
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/docarray/array/storage/elastic/backend.py", line 217, in _update_offset2ids_meta
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/elasticsearch/_sync/client/utils.py", line 414, in wrapped
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/elasticsearch/_sync/client/indices.py", line 1064, in exists
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 390, in perform_request
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 278, in perform_request
  File "/Users/jinadev/opt/anaconda3/envs/venv-now/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 274, in mimetype_header_to_compat
ModuleNotFoundError: import of re halted; None in sys.modules

Not clear whether this is a bug in docarray or elasticsearch. See discussion.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions