Skip to content

docs: add colab notebook for interactive 3d data visualization #749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoanFM merged 6 commits into from
Nov 11, 2022
Merged

docs: add colab notebook for interactive 3d data visualization #749

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f75e027
docs: add colab notebook for 3d data visualization
anna-charlotte Nov 10, 2022
ca69b57
docs: add colab notebook link
anna-charlotte Nov 10, 2022
7871d0d
Merge branch 'main' into docs-colab-3d-visualization-715
anna-charlotte Nov 11, 2022
a443070
docs: add 3d visualization notebook
anna-charlotte Nov 11, 2022
04fb1bb
docs: update output of stored notebook
anna-charlotte Nov 11, 2022
5ffb753
Merge branch 'main' into docs-colab-3d-visualization-715
anna-charlotte Nov 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions docs/datatypes/mesh/docarray_3d_data_visualization.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "LSPs1SsSZo-9"
},
"source": [
"# Interactive visualization of 3D data with DocArray\n",
"\n",
"## Supported representations\n",
"\n",
"DocArray supports the following representations for 3D data:\n",
"- Point cloud representation\n",
"- Vertices and faces representation\n",
"\n",
"In this notebook we want to demontrate how to visualize 3D data with some toydata examples that we prepared and stored for you in DocumentArrays in Jina Cloud."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VwAlK2NAadXk"
},
"source": [
"# Install requirements"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "YvlWf4QCafjn"
},
"outputs": [],
"source": [
"%%capture\n",
"!pip install git+https://github.com/docarray/docarray\n",
"!pip install docarray trimesh"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Y8uO3UfFzm0-"
},
"source": [
"# Login to Jina Cloud\n",
"\n",
"You need to login to Jina Cloud to access the 3D data stored in DocumentArrays which will be used in the course of this notebook. After logging in you will be able to pull those DocumentArrays from Jina Cloud and explore the stored data.\n",
"\n",
"If you don't have an account yet, you can [sign up here](https://jina-ai.us.auth0.com/u/signup?state=hKFo2SBVZEtMQVZVSlhXcXR2eVFOWTN2QnR6UmRhb19qUUd5WKFur3VuaXZlcnNhbC1sb2dpbqN0aWTZIF9VZFFRQjJnT24wdkQ1aFU2YkRxd2FBWm5XcHAtVXNio2NpZNkgN3BYQVVBdGlScXJ1TmQ2S0o2VTNaZDl1aGs1b0xxWkE)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "svA3onmsgwKI"
},
"outputs": [],
"source": [
"import hubble\n",
"\n",
"# Use Personal Access Token or browser to login. Token will be saved locally.\n",
"hubble.login()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "R9wmFqyZ1yOn"
},
"source": [
"# Point cloud representation\n",
"A point cloud is a representation of a 3D object. It is made by repeatedly and uniformly sampling points within the surface of the 3D body. It is a fixed size ndarray of shape (n_samples, 3). In DocArray this ndarray is stored in a Documents `.tensor` attribute. \n",
"\n",
"Here are some examples for point clouds of size 30,000 stored in a DocumentArray called 'da_point_clouds_30000' in Jina Cloud:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MJ_CGg483lpt"
},
"outputs": [],
"source": [
"from docarray import Document, DocumentArray\n",
"\n",
"da = DocumentArray().pull('da_point_clouds_30000')\n",
"da.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fmkFRVo2BnDA"
},
"source": [
"Next, let's visualize the point clouds interactively:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NXz8SeIs3-tB"
},
"outputs": [],
"source": [
"from IPython.display import Javascript\n",
"display(Javascript('''google.colab.output.setIframeHeight(0, true, {maxHeight: 5000})'''))\n",
"\n",
"for doc in da:\n",
" doc.summary()\n",
" doc.display()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ps5r-JCi4RR1"
},
"source": [
"# Point clouds of different densities\n",
"\n",
"In another DocumentArray we prepared some point cloud tensors of different sizes ranging from 2000 sampled datapoints up to 60000. For each Document we additionally stored the sample size value in the corresponding `.tags` dictionary. In the following you can choose the 3d object and the number of sampled points and display the corresponding point cloud."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "nvWJ0p4lkv8b"
},
"outputs": [],
"source": [
"from docarray import Document, DocumentArray\n",
"\n",
"sample_size = 100000 #@param [\"1000\", \"5000\", \"10000\", \"20000\", \"40000\", \"60000\", \"100000\"] {type:\"raw\"}\n",
"object_3d = \"mesh_man\" #@param [\"mesh_man\", \"flower\", \"skyscraper\"]\n",
"\n",
"da = DocumentArray().pull(f'da_point_cloud_different_densities_{object_3d}')\n",
"\n",
"for doc in da:\n",
" if doc.tags['sample size'] == sample_size:\n",
" doc.display()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-i7BelCPac38"
},
"source": [
"# Vertices and faces representation\n",
"\n",
"A 3D mesh can be represented by its vertices and faces. \n",
"Vertices are points in a 3D space, represented as a tensor of shape (n_points, 3). \n",
"Faces are triangular surfaces that can be defined by three points in 3D space, corresponding to the three vertices of a triangle. Faces can be represented as a tensor of shape (n_faces, 3). Each number in that tensor refers to an index of a vertex in the tensor of vertices.\n",
"\n",
"In DocArray, the vertices and faces of a mesh can be loaded and saved to a Document's `.chunks` by calling `.load_uri_to_vertices_and_faces()` on a Document instance.\n",
"\n",
"Again, we prepared a DocumentArray for you which we can now pull from Jina Cloud and interactivley visualize:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PyP3dR0eGkND"
},
"outputs": [],
"source": [
"from docarray import Document, DocumentArray\n",
"\n",
"da = DocumentArray().pull('da_vertices_and_faces_without_uri')\n",
"da.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yek-ihxpHu1J"
},
"outputs": [],
"source": [
"from IPython.display import Javascript\n",
"display(Javascript('''google.colab.output.setIframeHeight(0, true, {maxHeight: 5000})'''))\n",
"\n",
"for doc in da:\n",
" doc.summary()\n",
" doc.display()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "TIJnyFVqHz-_"
},
"outputs": [],
"source": []
}
],
"metadata": {
"colab": {
"provenance": [],
"collapsed_sections": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
9 changes: 8 additions & 1 deletion docs/datatypes/mesh/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,14 @@ This feature requires `trimesh`. You can install it via `pip install "docarray[f

A 3D mesh is the structural build of a 3D model consisting of polygons. Most 3D meshes are created via professional software packages, such as commercial suites like Unity, or the free open source Blender 3D.

DocArray supports .obj, .glb and .ply files.
DocArray supports the following file formats for 3D data handling: .obj, .glb and .ply.

You can explore interactive 3D data visualization with DocArray in the following Google Colab Notebook:
<p>
<a target="_blank" href="https://colab.research.google.com/drive/17uHgNVndRlfAAV9CAB_HEHHuRdGgsBW2?usp=sharing">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>
</p>

## Vertices and faces representation

Expand Down