contrib: Check symbols in the bitcoinconsensus shared library#25020
Closed
hebasto wants to merge 4 commits intobitcoin:masterfrom
Closed
contrib: Check symbols in the bitcoinconsensus shared library#25020hebasto wants to merge 4 commits intobitcoin:masterfrom
bitcoinconsensus shared library#25020hebasto wants to merge 4 commits intobitcoin:masterfrom
Conversation
Member
Yes, very good idea, Concept ACK. |
laanwj
reviewed
May 3, 2022
| def check_exported_symbols(binary) -> bool: | ||
| ok: bool = True | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY: | ||
| return ok |
Member
There was a problem hiding this comment.
I think one of the most important things to check for shared libraries is that they export the right symbols (to make sure their entire API is available), and only the right symbols (extra and re-exported symbols can cause conflicts).
Member
There was a problem hiding this comment.
Yea, that is what I assumed this would be doing when I first looked. Otherwise I think the checks currently being added are ~0.
Member
Author
There was a problem hiding this comment.
Exactly. It was the initial idea of this PR. When I faced with a crowd of exported symbols, I decided to make this PR focused on introducing a tool, and leave fighting with linker issues for follow ups.
Member
There was a problem hiding this comment.
I think the PR should be renamed then? If it's not actually doing the thing in the PR title.
fanquake
reviewed
May 3, 2022
| ok: bool = True | ||
|
|
||
| for symbol in binary.imported_symbols: | ||
| for symbol in binary.concrete.imported_symbols: |
Member
There was a problem hiding this comment.
2012abe: contrib: Always explicitly cast lief.Binary object
Why?
Member
There was a problem hiding this comment.
I also wonder, it's a bit ugly to have .concrete everywhere.
laanwj
reviewed
May 31, 2022
| print(f'{split[-1]} is not in ALLOWED_LIBRARIES!') | ||
| for dylib in binary.concrete.libraries: | ||
| library = dylib.name.split('/')[-1] | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY and library == 'libbitcoinconsensus.0.dylib': |
Member
There was a problem hiding this comment.
Let's not hardcode this name here. If we need to skip this, please define another constant for the list of libraries to skip.
Member
There was a problem hiding this comment.
In general, I don't really like how these changes are mixing up the binary and library checks.
Member
|
Speaking of checking exported symbols, I checked Most are related to C++ standard library (like vector), it seems? I wonder if there's a way to avoid this. |
Member
Author
It appears, #24994 fixes it: |
Member
That's beautiful! |
Contributor
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
Contributor
|
GUIX hashes x86: arm64: |
fanquake
reviewed
Dec 5, 2022
Member
fanquake
left a comment
fanquake
left a comment
There was a problem hiding this comment.
Is this change (going to be) dependant on #24994, or not? If so, maybe mark as a draft for now, and add that to the PR description?
Looks like there's still review comments outstanding, i.e here: https://github.com/bitcoin/bitcoin/pull/25020/files#r885641952 and here: https://github.com/bitcoin/bitcoin/pull/25020/files#r885637804.
| check-symbols: $(bin_PROGRAMS) $(LIBBITCOINCONSENSUS) | ||
| @echo "Running symbol and dynamic library checks..." | ||
| $(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/symbol-check.py $(bin_PROGRAMS) | ||
| $(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/symbol-check.py $(bin_PROGRAMS) $(builddir)/.libs/$$(echo $(LIBBITCOINCONSENSUS) | sed "s/.la//")$(LIBEXT) |
Member
There was a problem hiding this comment.
Is there a nicer way to do this? Would rather not re-introduce sed.
|
|
||
| self.assertEqual(call_symbol_check(cc, source, executable, ['-lexpat', '-Wl,-platform_version','-Wl,macos', '-Wl,11.4', '-Wl,11.4']), | ||
| (1, 'libexpat.1.dylib is not in ALLOWED_LIBRARIES!\n' + | ||
| (1, f'{executable}: libexpat.1.dylib is not in ALLOWED_LIBRARIES!\n' + |
Member
There was a problem hiding this comment.
I don't think adding the executable name to every line is an improvement. It's already printed in the failure.
| print(f'{split[-1]} is not in ALLOWED_LIBRARIES!') | ||
| for dylib in binary.concrete.libraries: | ||
| library = dylib.name.split('/')[-1] | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY and library == 'libbitcoinconsensus.0.dylib': |
Member
There was a problem hiding this comment.
In general, I don't really like how these changes are mixing up the binary and library checks.
| def check_exported_symbols(binary) -> bool: | ||
| ok: bool = True | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY: | ||
| return ok |
Member
There was a problem hiding this comment.
I think the PR should be renamed then? If it's not actually doing the thing in the PR title.
Member
Author
Drafted. |
fanquake
reviewed
Feb 14, 2023
| 'IPHLPAPI.DLL', # IP helper API | ||
| 'KERNEL32.dll', # win32 base APIs | ||
| 'msvcrt.dll', # C standard library for MSVC | ||
| 'libssp-0.dll', # mingw-w64 stack smashing protection |
Member
There was a problem hiding this comment.
NACK. This is accomodating a bug (in the DLL), and doing so in such a way, that it could leak into the actual binaries.
Member
Author
There was a problem hiding this comment.
This is accomodating a bug...
Which one?
Member
Author
Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In the light of a new shared library it looks reasonable to start checking shared library symbols.
During testing this PR, it was pointed that
libbitcoinconsensus.sohas "a ton of unnecessary symbols being exported". Fortunately, it could be fixed in #24994.Therefore, it makes sense to draft this PR until #24994 is merged.
Guix builds on
x86_64: