Skip to content

docs: add weaviate minimum working example #206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hanxiao merged 1 commit into from
Mar 15, 2022
Merged

docs: add weaviate minimum working example #206

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions docs/advanced/document-store/weaviate.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,69 @@ The following configs can be set:
| `protocol` | protocol to be used. Can be 'http' or 'https' | 'http' |
| `name` | Weaviate class name; the class name of Weaviate object to presesent this DocumentArray | None |
| `serialize_config` | [Serialization config of each Document](../../fundamentals/document/serialization.md) | None |

## Minimum Example

The following example shows how to use DocArray with Weaviate Document Store in order to index and search text
Documents.

First, let's run the create the `DocumentArray` instance (make sure a Weaviate server is up and running):

```python
from docarray import DocumentArray

da = DocumentArray(
storage="weaviate",
config={
"name": "Persisted",
"host": "localhost",
"port": 8080}
)
```

Then, we can index some Documents:

```python
from docarray import Document

da.extend([
Document(text='Persist Documents with Weaviate.'),
Document(text='And enjoy fast nearest neighbor search.'),
Document(text='All while using DocArray API.'),
])
```

Now, we can generate embeddings inside the database using BERT model:

```python
from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = AutoModel.from_pretrained('bert-base-uncased')

def collate_fn(da):
return tokenizer(
da.texts,
return_tensors='pt',
truncation=True,
padding=True
)

da.embed(model, collate_fn=collate_fn)
```


Finally, we can query the database and print the results:

```python
results = da.find(
DocumentArray([Document(text='How to persist Documents')]).embed(model, collate_fn=collate_fn),
limit=1
)

print(results[0].text)
```

```text
Persist Documents with Weaviate.
```