GStreamer plugins that bring SAHI slicing to NVIDIA DeepStream. The project keeps slicing, inference, and merge steps inside the DeepStream pipeline, using nvinfer for TensorRT execution and NvDsObjectMeta for post-processing.

This repository provides two plugins:

• nvsahipreprocess: computes frame slices, crops them on GPU, and prepares the input for nvinfer
• nvsahipostprocess: merges overlapping detections produced at slice boundaries using GreedyNMM

Typical pipeline:

nvstreammux -> nvsahipreprocess -> nvinfer -> nvsahipostprocess -> nvtracker -> nvdsosd

Most SAHI integrations around DeepStream run outside the pipeline, often in Python. This project keeps the workflow inside GStreamer so it can work with standard DeepStream components such as tracking, analytics, message brokers, and display elements.

Key points:

• SAHI slicing implemented as DeepStream plugins
• TensorRT inference handled by nvinfer
• Support for DeepStream 8.x and 9.x
• Test scripts and sample models included in the repository

The install.sh script detects the installed DeepStream version for Python bindings, builds the SAHI GStreamer plugins, and builds and installs libnvds_infer_yolo.so from deepstream_source/libs/nvdsinfer_yolo. That library is required to run the bundled ONNX models: they use TensorRT’s EfficientNMS post-processing, and this project’s parser decodes that output (it is not NVIDIA’s stock sample parser). Core TensorRT execution still uses the SDK’s libnvds_infer.so with nvinfer (not rebuilt here). Licensing: see LICENSE at the repository root and in deepstream_source/libs/nvdsinfer_yolo/.

This repository uses Git LFS for ONNX model files.

git lfs install
git clone https://github.com/levipereira/deepstream-sahi.git
cd deepstream-sahi

Run a DeepStream container:

docker run -it --name deepstream-sahi --net=host --gpus all \
    -v `pwd`:/apps/deepstream-sahi \
    -w /apps/deepstream-sahi \
    nvcr.io/nvidia/deepstream:9.0-triton-multiarch

Inside the container:

/apps/deepstream-sahi/install.sh
source /opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/pyds/bin/activate
cd /apps/deepstream-sahi/python_test/deepstream-test-sahi
python3 deepstream_test_sahi.py --model visdrone-full-640 --no-display --csv -i ../videos/aerial_crowding_01.mp4

Test videos are available on Google Drive and should be placed in python_test/videos/.

For container variants, display notes, rebuild mode, and environment details, see docs/INSTALL.md.

deepstream-sahi/
├── deepstream_source/
│   ├── gst-plugins/
│   │   ├── gst-nvsahipreprocess/
│   │   └── gst-nvsahipostprocess/
│   └── libs/
│       └── nvdsinfer_yolo/
├── python_test/
│   ├── common/
│   ├── deepstream-test-sahi/
│   └── videos/
├── train_yolov9_visdrone/
├── test_results/
├── scripts/
│   └── test_postprocess_params.sh
├── docs/
├── install.sh
└── README.md

• computes slice windows for each frame
• crops and rescales slices with NvBufSurfTransform
• forwards the resulting data to nvinfer

• reads detections from NvDsObjectMeta
• merges duplicates created by overlapping slices using a two-phase GreedyNMM algorithm
• supports IoU and IoS based matching with spatial hash grid indexing
• merges instance-segmentation masks (element-wise maximum)
• supports multiple GIE targeting (gie-ids="1;3;5")
• parallel per-frame processing via OpenMP
• configurable merge strategy (union / weighted / largest)

• nvinfer / libnvds_infer.so: from the DeepStream SDK (unchanged by this repo).
• libnvds_infer_yolo.so: built from this repository’s deepstream_source/libs/nvdsinfer_yolo/. Not optional for the default pipelines: the shipped models are exported with EfficientNMS (EfficientNMS_TRT / related ops), and this custom parser implements the bounding-box decoding for that layout. The sample PGIE configs set custom-lib-path to .../libnvds_infer_yolo.so.

Test setup:

• input video resolution: 2560x1440
• GPU: NVIDIA RTX 5080
• precision: FP16
• batch size: 16

For the complete benchmark, see docs/TEST_RESULTS.md.

Dense Pedestrian Crowd	Very Dense Crowd	Dense Vehicle Traffic

The repository includes both full-frame and slice-oriented training artifacts. The current results indicate that SAHI allows smaller model inputs to recover object scale on high-resolution video, which can help balance accuracy and throughput.

Training details are documented in docs/TRAINING.md.

All nvsahipostprocess parameters have been validated with automated tests across two density regimes:

Key findings:

• match-metric: IoS suppresses more duplicates than IoU (recommended for SAHI)
• match-threshold: monotonic — lower threshold → more aggressive suppression
• class-agnostic=true: +36% more suppression on vehicles, +13% on dense crowds
• enable-merge=false: reliably produces zero merges (pure NMS mode)
• max-detections: exact cap — removes 789 extra detections in dense scenes
• PERF profiling: GST_DEBUG=nvsahipostprocess:4 shows latency summary every ~1s

The postprocess NMM is never the bottleneck — TensorRT inference on 9 slices dominates. At 4× more detections, postprocess latency scales sub-linearly (spatial grid indexing).

Run the automated test suite:

# Default video (aerial_vehicles.mp4)
scripts/test_postprocess_params.sh

# Custom video
scripts/test_postprocess_params.sh python_test/videos/aerial_crowding_02.mp4

Full results: Parameter Tests — Vehicles | Parameter Tests — Dense Crowd

• The bidirectional NMM algorithm (non-greedy, transitive merge chains) is not implemented. GreedyNMM covers real-time use-cases adequately.
• Merged mask resolution is capped at 512x512 to prevent excessive memory allocation.
• Only single-source pipelines have been validated end-to-end; multi-source is supported via OpenMP parallelism but has not been benchmarked.

See docs/PLUGINS.md for the full property reference and algorithm details.

The project is distributed under the terms of the NVIDIA DeepStream SDK License Agreement. See the LICENSE file at the repository root.

Some components carry additional notices in source headers (for example, derivative works of NVIDIA DeepStream samples in gst-nvsahipreprocess, or third-party copyright lines in python_test/common/). Preserve those notices when redistributing.