This is a demonstration of integration of Nvidia GPUs to self-managed Elastic Cloud on Kubernetes (ECK) for the purpose of acceleration of embeddings and indexing.
- Jupyter notebook
- Builds an ECK deployment on Google Kubernetes Engine (GKE)
- GKE deployment includes CPU-only nodes for the Elastic Master nodes and CPU + Nvidia GPU nodes for the Elastic Data nodes.
- Creates a synthetic multi-lingual dataset with a text field and dense vector field from jina-embeddings-v3
- Executes a semantic search against that multi-lingual dataset
- Deletes the entire GKE environment
- GCP project
- gcloud CLI
- Elastic Cloud Connected API Key
- Python
- Create a Python virtual environment
- Execute notebook
- Elastic credentials will be stored in a .env file that is created dynamically. Use those credentials to access Kibana.