Skip to content

Weaviate Minimum example failing #433

@ghpkishore

Description

@ghpkishore

Hi in your documentation: https://docarray.jina.ai/advanced/document-store/weaviate/#minimum-example for some reason, when i run using debug mode, it works perfectly, but fails when i run it all at once it fails.

I am using Macbook pro m1 chip.

I installed all the required packages as well as given in the example.

This is my code:

from docarray import DocumentArray
from transformers import AutoModel, AutoTokenizer

da = DocumentArray(
    storage="weaviate", config={"name": "Persisted", "host": "localhost", "port": 8080}
)

from docarray import Document

da.extend(
    [
        Document(text='Persist Documents with Weaviate.'),
        Document(text='And enjoy fast nearest neighbor search.'),
        Document(text='All while using DocArray API.'),
    ]
)


tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = AutoModel.from_pretrained('bert-base-uncased')



def collate_fn(da):
    return tokenizer(da.texts, return_tensors='pt', truncation=True, padding=True)


da.embed(model, collate_fn=collate_fn)



results = da.find(
    DocumentArray([Document(text='How to persist Documents')]).embed(
        model, collate_fn=collate_fn
    ),
    limit=1,
)


print(results[0].texts)

Output:

Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias']

  • This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  • This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
    /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/docarray/array/mixins/embed.py:96: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
    batch_inputs[k] = torch.tensor(v, device=device)
    ['Persist Documents with Weaviate.']
    Exception ignored in: <function BaseGetSetDelMixin.del at 0x11e743370>
    Traceback (most recent call last):
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/docarray/array/storage/base/getsetdel.py", line 290, in del
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/docarray/array/storage/weaviate/getsetdel.py", line 79, in _save_offset2ids
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/docarray/array/storage/weaviate/backend.py", line 226, in _update_offset2ids_meta
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/weaviate/data/crud_data.py", line 124, in create
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/weaviate/connect/connection.py", line 314, in post
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 590, in post
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 532, in request
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 711, in merge_environment_settings
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/utils.py", line 797, in get_environ_proxies
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/utils.py", line 781, in should_bypass_proxies
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 2649, in proxy_bypass
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 2626, in proxy_bypass_macosx_sysconf
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 2568, in _proxy_bypass_macosx_sysconf
    ImportError: sys.meta_path is None, Python is likely shutting down

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions