Skip to content

adjust checkpoint filters when you want to retrieve all events#380

Merged
tora-kozic merged 4 commits intomainfrom
fix/append-risk-severity-filter
Jul 6, 2022
Merged

adjust checkpoint filters when you want to retrieve all events#380
tora-kozic merged 4 commits intomainfrom
fix/append-risk-severity-filter

Conversation

@tora-kozic
Copy link
Copy Markdown
Contributor

Appends a riskSeverity.exists() filter in the case of using a checkpoint with just the --include-non-exposure option (a query to retrieve all events).

Going to add more descriptive error message for an INVALID_PAGE_TOKEN 400 error to py42 and should automatically be raised here.

@tora-kozic tora-kozic force-pushed the fix/append-risk-severity-filter branch from afb1510 to 6fecdd7 Compare July 5, 2022 17:41
return checkpoint
if checkpoint is not None:
return checkpoint
return ""
Copy link
Copy Markdown
Contributor Author

@tora-kozic tora-kozic Jul 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the query sends None as a page token, the nextPgToken is returned as null. You have to pass an empty string as the token to get the first page.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move this logic into _get_all_file_events as we want to pass the empty string instead of None even when --use-checkpoint isn't passed, as that would cap the results of any non-checkpoint query at 10k results.

@tora-kozic tora-kozic merged commit 1969575 into main Jul 6, 2022
@tora-kozic tora-kozic deleted the fix/append-risk-severity-filter branch November 30, 2023 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants