Skip to content

305 add ccache to linux build#994

Merged
dabele merged 38 commits intomainfrom
305-ccache-in-CI
May 30, 2024
Merged

305 add ccache to linux build#994
dabele merged 38 commits intomainfrom
305-ccache-in-CI

Conversation

@dabele
Copy link
Copy Markdown
Member

@dabele dabele commented Apr 5, 2024

Changes and Information

  • added ccache to CI linux builds (other builds maybe added later after some observation)
  • simplified linux build options, removed unused options to make the cache key easier

To see that the cache is working correctly:

  • The ccache action reports loading the cache at the beginning of the build job and cache misses/hits at the end
  • stored cache files can be viewed in github -> actions -> caches; this development branch doesn't save caches anymore but there are some cache files around from my testing
  • I tested the cache on the development branch before I added the restriction to only save on the main branch. compile times went down from ~15 mins to ~5 mins. this is with identical code, i.e. 100% cache hits. Compile times should be checked after a few nightly builds to see if everything works there.

Closes #305

Merge Request - Guideline Checklist

Please check our git workflow. Use the draft feature if the Pull Request is not yet ready to review.

Checks by code author

  • Every addressed issue is linked (use the "Closes #ISSUE" keyword below)
  • New code adheres to coding guidelines
  • No large data files have been added (files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
  • Tests are added for new functionality and a local test run was successful (with and without OpenMP)
  • Appropriate documentation for new functionality has been added (Doxygen in the code and Markdown files if necessary)
  • Proper attention to licenses, especially no new third-party software with conflicting license has been added
  • (For ABM development) Checked benchmark results and ran and posted a local test above from before and after development to ensure performance is monitored.

Checks by code reviewer(s)

  • Corresponding issue(s) is/are linked and addressed
  • Code is clean of development artifacts (no deactivated or commented code lines, no debugging printouts, etc.)
  • Appropriate unit tests have been added, CI passes, code coverage and performance is acceptable (did not decrease)
  • No large data files added in the whole history of commits(files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
  • On merge, add 2-5 lines with the changes (main added features, changed items, or corrected bugs) to the merge-commit-message. This can be taken from the briefly-list-the-changes above (best case) or the separate commit messages (worst case).

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.23%. Comparing base (575a146) to head (5c47821).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #994   +/-   ##
=======================================
  Coverage   96.23%   96.23%           
=======================================
  Files         128      128           
  Lines       10852    10852           
=======================================
  Hits        10443    10443           
  Misses        409      409           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mknaranja
Copy link
Copy Markdown
Member

@dabele Who has been involved here and who is supposed to do the review?

@dabele
Copy link
Copy Markdown
Member Author

dabele commented Apr 24, 2024

@dabele Who has been involved here and who is supposed to do the review?

it's not done yet. but I need the CI to run for testing this, so it's not a draft.

@dabele dabele requested a review from reneSchm May 29, 2024 15:21
Copy link
Copy Markdown
Member

@reneSchm reneSchm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very nice, I do have a few questions (and one suggestion) though.

A little off topic, but does this already help with the boost dependency, or do we need another solution for that? It currently takes up a lot of CI time, both for the download itself as well as for packing and transferring artifacts, as these pack the entire build directory.

Comment thread .github/actions/linux-build/action.yml
Comment thread .github/actions/linux-build/action.yml
Comment thread .github/actions/linux-build/action.yml
@dabele
Copy link
Copy Markdown
Member Author

dabele commented May 30, 2024

A little off topic, but does this already help with the boost dependency, or do we need another solution for that? It currently takes up a lot of CI time, both for the download itself as well as for packing and transferring artifacts, as these pack the entire build directory.

Sadly, this doesn't help with that. I though about caching boost, but this would only be possible if the same bootstrapped boost is usable by all builds (or at least all builds of the same operating system family). boost is more than 1G, so if every build needs it's own cached version this will exceed the limit. This will only help with the download and bootstrap phase of boost, not sure what could be done about the artifact size. I guess only saving artifacts that are really required, i.e. test executables and shared objects.

@dabele dabele merged commit 2994f76 into main May 30, 2024
@dabele dabele deleted the 305-ccache-in-CI branch May 30, 2024 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore using ccache for CI pipeline

3 participants