Skip to content

Bbox score filtering#3118

Closed
tlancaster6 wants to merge 2 commits intoDeepLabCut:mainfrom
tlancaster6:bbox_score_filtering
Closed

Bbox score filtering#3118
tlancaster6 wants to merge 2 commits intoDeepLabCut:mainfrom
tlancaster6:bbox_score_filtering

Conversation

@tlancaster6
Copy link
Copy Markdown
Contributor

Summary: Filters out low-confidence bounding box detections during top-down pose estimation to reduce false-positives

I'm working on a project where the animals frequently leave the field of view for extended periods. While I found that top-down pose estimation worked well when both of my animals were in frame, it also resulted in false-positives when less than the max number of animals were in frame. For example, when only one of the two animals was in frame, I would see the scenario shown in the attached image, where two bounding boxes would be placed over the same individual and some keypoints would be detected twice. Upon further investigation, I found that the "bad" bounding boxes almost always had very low detection confidences. So I implemented a new RemoveLowConfidenceBoxes (inheriting from the PostProcessor class) in postprocessor.py, and integrated it into the build_detector_postprocessor function. Now detections with low confidence are removed before reaching the pose estimation phase, preventing the issue. Feedback and suggestions welcome!

image

@AlexEMG AlexEMG requested a review from maximpavliv October 14, 2025 09:34
@AlexEMG AlexEMG requested a review from deruyter92 November 18, 2025 23:38
@deruyter92
Copy link
Copy Markdown
Collaborator

deruyter92 commented Nov 19, 2025

@tlancaster6, thanks for your contribution, looks like a useful addition! I have two minor suggestions:

  1. Instead of the fixed threshold of 0.25, it would be good to have an adjustable value with the default behavior that no filtering is applied. This preserves backwards compatibility (and reproducibility).
  2. There was a small issue with the following part of your code
if len(output) > len(keepers):
    predictions[name] = output[keepers]

Since np.where returns a tuple with indices, the length of keepers does not reflect the number of bounding boxes that is above threshold. I propose to change it to:

above_threshold = predictions["bbox_scores"] > self.bbox_score_thresh
keepers = np.where(above_threshold)
if any(~above_threshold):
    predictions['bboxes'] = predictions["bboxes"][keepers]
    predictions['bbox_scores'] = predictions["bbox_scores"][keepers]

I have requested some proposed changes. Please have a look if you agree and if it works for your solution.

Copy link
Copy Markdown
Collaborator

@deruyter92 deruyter92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding this feature could be definitely valuable. With regards to the implementation I suggested some changes. Since I cannot directly contribute to this branch, I've opened a replacement PR #3154.

@deruyter92
Copy link
Copy Markdown
Collaborator

This PR is closed in favor of #3154

@deruyter92 deruyter92 closed this Jan 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants