Skip to content

Feature/436 num chains#492

Merged
mitzimorris merged 40 commits intodevelopfrom
feature/436-num-chains
Nov 11, 2021
Merged

Feature/436 num chains#492
mitzimorris merged 40 commits intodevelopfrom
feature/436-num-chains

Conversation

@mitzimorris
Copy link
Member

@mitzimorris mitzimorris commented Nov 5, 2021

Submission Checklist

  • Run unit tests
  • Declare copyright holder and open-source license: see below

Summary

Added logic to expose CmdStan 2.28 cross-chain parallelism via argument num_chains.
When model has been compiled with STAN_THREADS=true, sample method will use num_chains arg to run multi-chain process by default.

  • Refactored RunSet object so that is has attributes for both number of subprocesses and number of chains. Removed redundant checks on input args in RunSet; documented RunSet class accordingly.

  • Added tqdm hook to handle multi-chain processes output.

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

@mitzimorris mitzimorris requested a review from WardBrian November 5, 2021 02:27
@mitzimorris mitzimorris marked this pull request as draft November 5, 2021 02:28
@mitzimorris
Copy link
Member Author

Setting up PR. more unit tests would be good.

Copy link
Member

@WardBrian WardBrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for doing this so quickly

@mitzimorris
Copy link
Member Author

thanks for reviewing this so quickly. great feedback - will address over the weekend.

@mitzimorris
Copy link
Member Author

regarding CmdStan args vs. method-specific args, the CmdStan CLI requires that args be grouped under headings "method", "data", "output". Given this, you're correct that num_chains should be part of the sampler args.

@codecov-commenter
Copy link

codecov-commenter commented Nov 9, 2021

Codecov Report

Merging #492 (79c1df6) into develop (725ad89) will increase coverage by 0.44%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #492      +/-   ##
===========================================
+ Coverage    77.95%   78.40%   +0.44%     
===========================================
  Files           30       30              
  Lines         8991     9186     +195     
===========================================
+ Hits          7009     7202     +193     
- Misses        1982     1984       +2     
Impacted Files Coverage Δ
a/cmdstanpy/cmdstanpy/cmdstanpy/stanfit.py 92.28% <0.00%> (-0.19%) ⬇️
cmdstanpy/cmdstanpy/stanfit.py 92.51% <0.00%> (-0.18%) ⬇️
...nner/work/cmdstanpy/cmdstanpy/cmdstanpy/stanfit.py 92.51% <0.00%> (-0.18%) ⬇️
cmdstanpy/cmdstanpy/cmdstan_args.py 94.90% <0.00%> (+0.02%) ⬆️
...work/cmdstanpy/cmdstanpy/cmdstanpy/cmdstan_args.py 94.90% <0.00%> (+0.02%) ⬆️
a/cmdstanpy/cmdstanpy/cmdstanpy/cmdstan_args.py 94.09% <0.00%> (+0.02%) ⬆️
cmdstanpy/cmdstanpy/__init__.py 87.50% <0.00%> (+0.54%) ⬆️
a/cmdstanpy/cmdstanpy/cmdstanpy/__init__.py 87.50% <0.00%> (+0.54%) ⬆️
...ner/work/cmdstanpy/cmdstanpy/cmdstanpy/__init__.py 87.50% <0.00%> (+0.54%) ⬆️
a/cmdstanpy/cmdstanpy/cmdstanpy/utils.py 75.00% <0.00%> (+0.64%) ⬆️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 725ad89...79c1df6. Read the comment docs.

@mitzimorris
Copy link
Member Author

if all unit tests pass, this is ready for re-review.

@WardBrian
Copy link
Member

Great, I'll review after that change.

One thing that immediately strikes out to me: I think the info_dict method may be useful for other stuff in the future and might belong as part of the CmdStanModel class. That would also allow us to move the if exe_file is not None logic into the method itself, so we could just call model.info() and get back either the dictionary or None

@WardBrian
Copy link
Member

Actually, to add on to the above comment, we should make info_dict return {} instead of None. That way we can (always) safely do something like model.info_dict().get('STAN_THREADS') without checking the return first.

@mitzimorris
Copy link
Member Author

the code is ready for review, but we still need to update the doc and and examples for 2.28.

the changes made on the CmdStan side w/r/t fixed_params should make people's lives easier (even if the code ties itself in knots trying to check the console outputs and CSV files).

@WardBrian WardBrian marked this pull request as ready for review November 10, 2021 14:29
Copy link
Member

@WardBrian WardBrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the work on this. A few more nitpicky things and hopefully the last go-round on the progress bars

Copy link
Member

@WardBrian WardBrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is probably my last comments. This PR is looking great when you consider how inherently messy the cmdstan interface is

Copy link
Member

@WardBrian WardBrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - 1.0 here we come!

@mitzimorris
Copy link
Member Author

mitzimorris commented Nov 11, 2021

this totally sucks - discovered that model method compile with option force=True doesn't do the right thing and there are no unit tests for this either. wtf?

making this work is important in order to be able to explain how to take advantage of new num_threads. the problem seems to be that make won't update target unless it needs to. changed logic in compile method to remove the offending exe file, but then some unit tests are failing. perhaps because of implicit dependencies? investigating.

@WardBrian
Copy link
Member

Unless you believe that bug was introduced by this PR I think it’s better to merge this and handle that separately. These big PRs get very hard to review

@mitzimorris
Copy link
Member Author

this bug has been there all along - that's why unit tests are removing exe files here and there. I'm just surprised that this wasn't tested more back when we added this feature.

@mitzimorris mitzimorris merged commit 3cbfae4 into develop Nov 11, 2021
@mitzimorris mitzimorris deleted the feature/436-num-chains branch November 17, 2021 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants