Skip to content

Use os.sched_getaffinity for cpu counts when available#2979

Merged
asottile merged 1 commit intopre-commit:mainfrom
jdb8:cpu-sched-getaffinity
Aug 30, 2023
Merged

Use os.sched_getaffinity for cpu counts when available#2979
asottile merged 1 commit intopre-commit:mainfrom
jdb8:cpu-sched-getaffinity

Conversation

@jdb8
Copy link
Copy Markdown
Contributor

@jdb8 jdb8 commented Aug 29, 2023

Attempt at fixing/improving #2978 for systems that support the os.sched_getaffinity method.

When unavailable (i.e. on mac), we fall back to the old behaviour.

I didn't know how much to mess with the existing tests so went with a new fixture to simulate when this is unavailable - I don't have a windows machine to test on so let me know if there's a better way to avoid this change being annoying!

resolves #2978

@jdb8 jdb8 force-pushed the cpu-sched-getaffinity branch from 1b44cbd to 92f905a Compare August 29, 2023 18:10
@jdb8 jdb8 force-pushed the cpu-sched-getaffinity branch from 0ee30cd to 4165ed8 Compare August 30, 2023 05:44
@asottile asottile force-pushed the cpu-sched-getaffinity branch from 4165ed8 to ea8244b Compare August 30, 2023 17:41
@asottile
Copy link
Copy Markdown
Member

pre-commit.ci run

Copy link
Copy Markdown
Member

@asottile asottile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asottile asottile merged commit ac42dc5 into pre-commit:main Aug 30, 2023
@deronnax
Copy link
Copy Markdown

shimming in to tell you - in case you don't alreadu know it - that os.process_cpu_count was introduced in python 3.13 for this purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

CPU count logic does not distinguish "usable" CPUs inside cgroup'd containers (e.g. inside k8s)

3 participants