impl index spec by shivamka1 · Pull Request #2103 · Pometry/Raphtory

shivamka1 · 2025-05-22T17:24:55Z

This PR implements IndexSpec that can be used to create index for properties selectively: #1985

 let graph = init_graph(Graph::new());

let index_spec = IndexSpecBuilder::new(graph.clone())
    .with_const_node_props(vec!["x"])?
    .with_all_temp_node_props()
    .with_all_edge_props()
    .build();

graph.create_index_in_ram_with_spec(index_spec).unwrap();

On python this looks like:

graph = init_graph(Graph())
spec = (
    IndexSpecBuilder(graph)
    .with_const_node_props(["y"])
    .with_temp_node_props(["p1"])
    .with_const_edge_props(["e_y"])
    .with_temp_edge_props(["e_p1"])
    .build()
)

graph.create_index_in_ram_with_spec(spec)

…eated, add tests

github-actions

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: `425556f`	Previous: `b1a22cf`	Ratio
`lotr_graph_window_100/iterate nodes`	`7238` ns/iter (`± 449`)	`3110` ns/iter (`± 46`)	`2.33`

This comment was automatically generated by workflow using github-action-benchmark.

…ate apis

ljeub-pometry

Need to fix race condition in index creation

python/test_utils/filters_setup.py

python/tests/test_base_install/test_index_spec.py

raphtory/src/core/utils/errors.rs

raphtory/src/db/api/storage/storage.rs

raphtory/src/search/entity_index.rs

raphtory/src/search/graph_index.rs

raphtory/src/search/node_index.rs

ljeub-pometry

Some minor things to tidy up but looking good otherwise

python/test_utils/utils.py

python/tests/test_base_install/test_graphql/test_edge_sorting.py

python/tests/test_base_install/test_graphql/test_filters/test_graph_edges_property_filter.py

python/tests/test_base_install/test_graphql/test_filters/test_graph_nodes_property_filter.py

python/tests/test_base_install/test_graphql/test_filters/test_nodes_property_filter.py

raphtory/src/db/graph/views/filter/mod.rs

raphtory/src/python/graph/index.rs

raphtory/src/search/entity_index.rs

raphtory/src/search/mod.rs

raphtory/src/search/node_index.rs

shivamka1 added 3 commits May 22, 2025 18:24

impl index spec

d4485db

fallback to raphtory apis for properties for which indexes are not cr…

ce6e2d9

…eated, add tests

impl index spec for python

425556f

github-actions bot reviewed May 27, 2025

View reviewed changes

shivamka1 and others added 22 commits May 27, 2025 16:30

impl get index spec

94b2a61

fix loading index spec based index

9243580

move all index tests in one place

8675021

add test

16d1ead

impl py get_index_spec

f88b9cd

rw indexspec

aa570c4

check if index needs update

054cfde

impl update index

c07aab4

add test

5e17d7d

more test

d478677

update only existing prop index, don't add any new prop index via upd…

72add5a

…ate apis

add tests

ec8777a

Merge branch 'master' into features/index_spec

9912a76

fmt

ac08e60

fix issues with property indexes, add/fix tests

98aa850

ref

d1e0ac0

add update index to create_index_in_ram_with_spec, add test

397afb8

ref py tests

290908e

impl gql node type filtering for list of node types

ab59db1

add test

b0fe461

fix test

d08d0b1

ref

0953cd4

ljeub-pometry requested changes Jun 2, 2025

View reviewed changes

shivamka1 added 3 commits June 2, 2025 18:55

fix tests

165172c

rid UnknownProperty

199a086

add comment

908650e

shivamka1 added 5 commits June 2, 2025 20:53

rid name and prop type from index spec

ecb90ca

fix indexspec for python

d806701

fix race

5756687

impl suggestions from review

e4ab6aa

intro index_ops

4bc1adb

shivamka1 mentioned this pull request Jun 4, 2025

Deduplicate props in raphtory APIs #2112

Open

shivamka1 and others added 4 commits June 4, 2025 16:31

ref, fix warnings

a9cdf07

add test

b167a57

fix build

47143f2

Merge branch 'master' into features/index_spec

29b9b07

ljeub-pometry requested changes Jun 5, 2025

View reviewed changes

shivamka1 added 7 commits June 5, 2025 13:07

add test

780a36c

fix bench

b36c4e5

ref

d65b19b

ref

00bb1fe

add props to index_spec

6a6c27f

ref tests

a1aa852

fix warnings

f11ef8d

shivamka1 marked this pull request as ready for review June 5, 2025 15:12

ljeub-pometry approved these changes Jun 6, 2025

View reviewed changes

shivamka1 merged commit 48a104d into master Jun 6, 2025
28 of 29 checks passed

shivamka1 deleted the features/index_spec branch June 6, 2025 08:17

Conversation

shivamka1 commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

⚠️ Performance Alert ⚠️

Uh oh!

ljeub-pometry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ljeub-pometry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shivamka1 commented May 22, 2025 •

edited

Loading