The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #17919 (depends: Allow building with system clang by dongcarl)
• #15441 ([doc] build: warn against spaces in working directory by Sjors)
• #12557 ([WIP] 64 bit iOS device support by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Concept ACK

Tested building depends on macOS 10.14.6 Mojave. I still had to do this annoying workaround:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

And building protobuf fails: log

If I remove protobuf and native_protobuf it does build. I can then compile QT:

./autogen.sh
./configure --prefix=`pwd`/depends/x86_64-apple-darwin18.7.0 --disable-bip70
make

This works, but I get a flood of warnings at the end: log

Will test on macOS 10.15 Catalina later.

We should backport this to 0.18 given #16387 and ideally do a 0.18.2 tag before Catalina is out (September or October). Our 0.19 release is scheduled for November #15940, so otherwise there's no tag to build from.

Concept ACK on bumping macOS from 10.10 to 10.12, though a small percentage of users would have to stay on the 0.18 branch (the last tag before backport).

The bottleneck for upgrading macOS is hardware, and that's pretty durable. Here are the minimum specs for 10.10:

Minimum specs for 10.12:

Some refurbished machines for sale:

Some second hand machines at an even lower price point still work:

Some don't:

I still had to do this annoying workaround:

Yep, that isn't new with this change, but may go away if we can remove any reliance on having the headers in the now deprecated location. See details in the Command Line Tools section of the Xcode 10 release notes.

If I remove protobuf and native_protobuf it does build. I can then compile QT:

Did you try to build depends before installing the new headers? If you make clean in depends, then try rebuilding as usual it should work. Nothing Protobuf related is being changed in this PR.

Concept ACK.
Unsure about the timing, if its already the time to drop 10.10 & 10.11 since 10.11 was released on Oct 2015.
According to statista, in April 2019, 10.10 and 10.11 makes about 16% of mac installations.
(https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/)

Aidium publishes their update stats and it looks like, >10% was using 10.11 this week.
https://www.adium.im/sparkle/

It'd say we wait with this.

Did you try to build depends before installing the new headers? If you make clean in depends

I thought I did, but maybe not. I just tried make clean and now protobuf indeed just buids (on
536034b).

It'd say we wait with this.

If it doesn't break anything else, maybe we can do this bump when we bump QT to 5.12 (LTS)?

Updated to add some documentation (Linux) for the macOS 10.14 SDK extraction. This doesn't have to be the final docs, but should be useful for someone on a Linux machine wanting to extract the SDK and test this PR.

It'd say we wait with this.

We don't have to bump the minimum to 10.12 right now. We could do 10.11, or potentially leave it at 10.10 assuming there aren't any issues using the new SDK. Regardless, I'd like the rest of the changes (or at least the new Clang) to make it into v0.19.0, so that building for macOS isn't broken on Debian Buster #16052.

From Apple: "All Macintosh computers that can run Mountain Lion, Mavericks, or Yosemite can run El Capitan" (10.11). I suspect many people just don't realize they can upgrade.

I managed to build depends on macOS 10.14.6 (Mojave) with the minimum version set to 10.10

[...] may go away if we can remove any reliance on having the headers in the now deprecated location

That means this PR is not enough to support Catalina, at least not without some other workaround.

I tested a few of my cleanup suggestions above, and they seem to work fine.
Feel free to pull the top two from here: https://github.com/theuni/bitcoin/commits/16392, or adapt them into your commits however's easiest.

@theuni I've incorporated some of your suggestions (trimming the LD64 stuff, and the clang hacks) and added you as co-author to the relevant commits.

We are now pointing back to upstream libdmg-hfsplus, with a single patch included in depends.

We are also now pointing back to the upstream cctools-port repo.

libtapi is now being built with our 6.0.1 Clang and I've removed one of the intermediate directories I was creating.

I still need to address:

We want this built statically if possible, otherwise this introduces a (rather unnecessary) LDPATH runtime requirement. Does build.sh have static lib support?

Building depends on macOS 10.14.6 Mojave still works as of 97b3349.

Tested ACK via a depends build with macOS SDK 10.14:
https://bitcoinbuilds.org/?job=e969d43f-2cd8-4890-b5ae-bcc2d0ea4195

FWIW https://bitcoincore.org/depends-sources/sdks/MacOSX10.14.sdk.tar.gz is up

Downloaded file

$ sha256sum MacOSX10.14.sdk.tar.gz 
2530c1d88f1aca98994cf32cfcf6099d880965ef0200b8b2a87279c191d56766  MacOSX10.14.sdk.tar.gz

seems broken.

UPDATE: it was a download error. This one

$ sha256sum MacOSX10.14.sdk.tar.gz 
328aff47e28c17093d59a72712a9b2e62cd8a8b87bbe03f91abb32960b413f0f  MacOSX10.14.sdk.tar.gz

is ok.

Tested ca5055a

Cross compilation on Linux Mint 19.3 with depends and MacOSX10.14.sdk was contaminated by system Qt packages (#18042, #18045, #18051):

$ make HOST=x86_64-apple-darwin16 -C depends
$ ./autogen.sh
$ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin16/share/config.site ./configure
$ make
...
  CXX      qt/qt_libbitcoinqt_a-peertablemodel.o
qt/notificator.cpp:50:21: error: allocation of incomplete type 'QDBusInterface'
    interface = new QDBusInterface("org.freedesktop.Notifications",
                    ^~~~~~~~~~~~~~
...

The patch from #18051 was applied, then compilation passed successfully with loads of linker warnings like this:

  AR       qt/libbitcoinqt.a
  CXXLD    qt/bitcoin-qt
  CXXLD    qt/test/test_bitcoin-qt
ld: warning: direct access in function '__ZN5boost10filesystem6detail9canonicalERKNS0_4pathES4_PNS_6system10error_codeE' from file '/home/hebasto/GitHub/bitcoin/depends/x86_64-apple-darwin16/lib/libboost_filesystem-mt-x64.a(operations.o)' to global weak symbol '__ZZN5boost6system16generic_categoryEvE25generic_category_instance' from file 'libbitcoin_server.a(libbitcoin_server_a-init.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
...

The resulted Bitcoin-Core.dmg was successfully installed on macOS 10.14 then launched .

Gitian builds

@MarcoFalke Looks like DrahtBot is still missing the new SDK.

re-ACK 7e21044 (rebased from 248526e)

@fanquake I apologize for my lateness, but here's my outstanding changes so that we can generate and use the SDK with headers as requested: fanquake/bitcoin@macos-toolchain-update...dongcarl:2020-01-macos-sdk-with-headers

Here are the new instructions: https://github.com/dongcarl/bitcoin/blob/adf9968de5dc872e3c51e538f6b99e9423b65bd3/contrib/macdeploy/README.md#step-2-generating-macosx1014sdktargz-from-xcodeapp

Let's try to get this merged soon—if we still want this for 0.20 it's not something that should be done last minute.

ACK 7e21044

Immediate followup: #18072

Built depends successfully on 10.14.6

Is there anything more to say re the "obsolete mlinker-version option"? Is it obsolete only in that it is no longer required for build?

@fanquake I apologize for my lateness, but here's my outstanding changes so that we can generate and use the SDK with headers as requested: fanquake/[email protected]:2020-01-macos-sdk-with-headers

Thanks, please add these instructions in a new PR!

@fanquake I apologize for my lateness, but here's my outstanding changes so that we can generate and use the SDK with headers as requested: fanquake/[email protected]:2020-01-macos-sdk-with-headers

Thanks, please add these instructions in a new PR!

It's here! :-) #18072