Background
I am currently using Pixie to predict object parameters for objects from the Objaverse dataset. While the framework work for the provided pixieverse data, I encountered a bottleneck when attempting to run the pipeline on new, unseen objects (those not included in the pre-processed pixieverse set).
The Problem
When transitioning from voxelization to dataset construction for a new object, the system fails to build the MaterialVoxelDataset due to a missing file: obj_id/sample_0/material_grid.npy
According to my understanding, for inference on novel objects, Pixie should ideally only require multi-view images. However, the MaterialVoxelDataset logic seems to expect a pre-existing material grid.
Current Workaround
After experimenting with the pipeline, I found that the issue can be bypassed by treating these Objaverse objects similarly to "real-world scenes." Specifically, I modified the logic in the pipeline by adding format_real_scene_sample before the neural segmentation step:
# Original logic
# sample_output_dir = generate_neural_segmentation(cfg, render_output_dir, paths)
# Modified logic
format_real_scene_sample(cfg, paths) # Manually trigger the creation of an empty/placeholder material_grid
sample_output_dir = generate_neural_segmentation(cfg, render_output_dir, paths)
By adding this step, the pipeline generates the necessary (though potentially empty) material_grid.npy, allowing the data flow to complete and the inference to proceed.
Therefore, my question is that is this the intended or standard way to handle out-of-dataset objects from Objaverse?
Thank you for your time and for this outstanding framework!
Background
I am currently using Pixie to predict object parameters for objects from the Objaverse dataset. While the framework work for the provided pixieverse data, I encountered a bottleneck when attempting to run the pipeline on new, unseen objects (those not included in the pre-processed pixieverse set).
The Problem
When transitioning from voxelization to dataset construction for a new object, the system fails to build the MaterialVoxelDataset due to a missing file: obj_id/sample_0/material_grid.npy
According to my understanding, for inference on novel objects, Pixie should ideally only require multi-view images. However, the MaterialVoxelDataset logic seems to expect a pre-existing material grid.
Current Workaround
After experimenting with the pipeline, I found that the issue can be bypassed by treating these Objaverse objects similarly to "real-world scenes." Specifically, I modified the logic in the pipeline by adding format_real_scene_sample before the neural segmentation step:
By adding this step, the pipeline generates the necessary (though potentially empty) material_grid.npy, allowing the data flow to complete and the inference to proceed.
Therefore, my question is that is this the intended or standard way to handle out-of-dataset objects from Objaverse?
Thank you for your time and for this outstanding framework!