Skip to content

enable system SSL certificates in OpenSSL installations#2683

Merged
boegel merged 7 commits intoeasybuilders:developfrom
lexming:ssl-certs
Mar 4, 2022
Merged

enable system SSL certificates in OpenSSL installations#2683
boegel merged 7 commits intoeasybuilders:developfrom
lexming:ssl-certs

Conversation

@lexming
Copy link
Copy Markdown
Contributor

@lexming lexming commented Feb 18, 2022 •
edited by boegel
Loading

fixes #2674, fixes easybuilders/easybuild-easyconfigs#14901

Changelog:

  • make OpenSSL wrapper to error out instead of building OpenSSL from source if the only missing pieces in the system are the header files. On error, inform the user to either install the development package of openssl in the system or force the installation from source in EasyBuild with wrap_system_openssl
  • symlink system certificates (if present) in the installation directory of OpenSSL whenever it is build from source
  • add ssl_certificates option to openssl easyblock to define the path to system certificates
  • update sanity checks in openssl easyblock: remove hardcoded library extensions, remove checks for lib and lib64 (now EB installs both), add checks on ssl/certs whenever certificates are symlinked

The link to the system certificates is determined in the following order:

  • use ssl_certificates path if defined
  • fall back to OPENSSLDIR from system openssl
  • fall back to generic path in GENERIC_SSL_CERTS_DIR
  • fall back to installation without certificates and print warning informing the user

@lexming lexming added this to the next release (4.5.4?) milestone Feb 18, 2022
@boegelbot

This comment was marked as outdated.

Sign in to view
@lexming
Copy link
Copy Markdown
Contributor Author

lexming commented Feb 19, 2022 •
edited
Loading

Test report by @lexming

Overview of tested easyconfigs (in order)

  • SUCCESS OpenSSL-1.1.1b-GCCcore-8.2.0.eb
  • SUCCESS OpenSSL-1.1.1d-GCCcore-8.3.0.eb
  • SUCCESS OpenSSL-1.1.1e-GCCcore-9.3.0.eb
  • SUCCESS OpenSSL-1.1.1h-GCCcore-10.2.0.eb
  • SUCCESS OpenSSL-1.1.1k-GCCcore-10.3.0.eb
  • SUCCESS OpenSSL-1.1.1k-GCCcore-11.2.0.eb

Build succeeded for 6 out of 6 (6 easyconfigs in total)
node305.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz, Python 2.7.5
See https://gist.github.com/5e3788eff4e94ca74dcabd2ef88f9814 for a full test report.

OPENSSLDIR points to %(installdir)s/ssl (its default value).
The resulting ssl folder on CentOS 7

ssl/
├── certs -> /etc/pki/tls/certs
├── ct_log_list.cnf
├── ct_log_list.cnf.dist
├── misc
│   ├── CA.pl
│   ├── tsget -> tsget.pl
│   └── tsget.pl
├── openssl.cnf
├── openssl.cnf.dist
└── private

@lexming
Copy link
Copy Markdown
Contributor Author

lexming commented Feb 19, 2022 •
edited
Loading

Test report by @lexming

Overview of tested easyconfigs (in order)

  • SUCCESS OpenSSL-1.0.eb
  • SUCCESS OpenSSL-1.1.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node300.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz, Python 2.7.5
See https://gist.github.com/ff7513d04eb3c9d6e768c6b0bfb3f8f5 for a full test report.

Tests with default easyconfigs. OpenSSL in CentOS 7 is wrapped successfully. Nothing changes in this case. OPENSSLDIR correctly points to /etc/pki/tls

@lexming
Copy link
Copy Markdown
Contributor Author

lexming commented Feb 19, 2022 •
edited
Loading

Test report by @lexming

Overview of tested easyconfigs (in order)

  • SUCCESS OpenSSL-1.0.eb
  • SUCCESS OpenSSL-1.1.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node301.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz, Python 2.7.5
See https://gist.github.com/6641022a002257c9e7fa4ce1ba599a04 for a full test report.

Tests with wrap_system_openssl=False. OpenSSL is successfully built from source. OPENSSLDIR points to %(installdir)s/ssl (its default value).

ssl folder for v1.0 on CentOS 7

OpenSSL/1.0/ssl
├── certs -> /etc/pki/tls/certs
├── man
│   ├── man1
│   ├── man3
│   ├── man5
│   └── man7
├── misc
│   ├── CA.pl
│   ├── CA.sh
│   ├── c_hash
│   ├── c_info
│   ├── c_issuer
│   ├── c_name
│   └── tsget
├── openssl.cnf
└── private

ssl folder for v1.1 on CentOS 7

OpenSSL/1.1/ssl
├── certs -> /etc/pki/tls/certs
├── ct_log_list.cnf
├── ct_log_list.cnf.dist
├── misc
│   ├── CA.pl
│   ├── tsget -> tsget.pl
│   └── tsget.pl
├── openssl.cnf
├── openssl.cnf.dist
└── private

@boegel boegel changed the title Enable system SSL certificates in OpenSSL installations enable system SSL certificates in OpenSSL installations Mar 1, 2022
@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 4, 2022 •
edited
Loading

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenSSL-1.0.eb
  • SUCCESS OpenSSL-1.1.eb
  • SUCCESS OpenSSL-1.1.1k-GCCcore-11.2.0.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
node2682.swalot.os - Linux openSUSE Leap 15.3, x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.6.15
See https://gist.github.com/49637d370970760f83a921f8a3a83507 for a full test report.

$ grep PRETTY /etc/os-release
PRETTY_NAME="openSUSE Leap 15.3"
$ rpm -qa | grep ^openssl
openssl-1.1.1d-1.46.noarch
openssl-1_1-1.1.1d-11.38.1.x86_64

$ ls OpenSSL/1.0/ssl
certs  man  misc  openssl.cnf  private
$ ls -l OpenSSL/1.0/ssl/certs
lrwxrwxrwx 1 vsc40023 vsc40023 14 Mar  4 20:14 OpenSSL/1.0/ssl/certs -> /etc/ssl/certs

$ ls -l OpenSSL/1.1/bin/openssl OpenSSL/1.1/lib/libssl.so
lrwxrwxrwx 1 vsc40023 vsc40023 16 Mar  4 20:14 OpenSSL/1.1/bin/openssl -> /usr/bin/openssl
lrwxrwxrwx 1 vsc40023 vsc40023 13 Mar  4 20:14 OpenSSL/1.1/lib/libssl.so -> libssl.so.1.1

@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 4, 2022 •
edited
Loading

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenSSL-1.0.eb
  • SUCCESS OpenSSL-1.1.1d-GCCcore-8.3.0.eb
  • SUCCESS OpenSSL-1.1.1h-GCCcore-10.2.0.eb
  • SUCCESS OpenSSL-1.1.1k-GCCcore-10.3.0.eb
  • SUCCESS OpenSSL-1.1.1k-GCCcore-11.2.0.eb
  • SUCCESS OpenSSL-1.1.eb

Build succeeded for 6 out of 6 (6 easyconfigs in total)
node3502.doduo.os - Linux RHEL 8.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/6f4228a16c0a63d4865fbfc442bef896 for a full test report.

$ grep PRETTY /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
$ rpm -qa | grep ^openssl
openssl-devel-1.1.1g-15.el8_3.x86_64
openssl-libs-1.1.1g-15.el8_3.x86_64
openssl-1.1.1g-15.el8_3.x86_64
openssl-libs-1.1.1g-15.el8_3.i686

$ ls -l OpenSSL/1.0/ssl/certs
lrwxrwxrwx 1 vsc40023 vsc40023 18 Mar  4 21:30 OpenSSL/1.0/ssl/certs -> /etc/pki/tls/certs
$ ls -l OpenSSL/1.1.1k-GCCcore-11.2.0/ssl/certs
lrwxrwxrwx 1 vsc40023 vsc40023 18 Mar  4 21:49 OpenSSL/1.1.1k-GCCcore-11.2.0/ssl/certs -> /etc/pki/tls/certs

@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 4, 2022 •
edited
Loading

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenSSL-1.0.eb
  • SUCCESS OpenSSL-1.1.eb
  • SUCCESS OpenSSL-1.1.1k-GCCcore-11.2.0.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
node2682.swalot.os - Linux Ubuntu 20.04.3 LTS (Focal Fossa), x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.8.10
See https://gist.github.com/895ff8cd16466705083f7cd70611effc for a full test report.

$ grep PRETTY /etc/os-release
PRETTY_NAME="Ubuntu 20.04.3 LTS"
$ apt list openssl | grep installed
openssl/focal-updates,now 1.1.1f-1ubuntu2.10 amd64 [installed,automatic]

$ ls OpenSSL/1.0/ssl
certs  man  misc  openssl.cnf  private
$ ls -l OpenSSL/1.0/ssl/certs
lrwxrwxrwx 1 vsc40023 vsc40023 18 Mar  4 21:09 OpenSSL/1.0/ssl/certs -> /usr/lib/ssl/certs

$ ls -l OpenSSL/1.1/{bin,lib64}/{openssl,libssl.so} 2>/dev/null
lrwxrwxrwx 1 vsc40023 vsc40023 16 Mar  4 21:09 OpenSSL/1.1/bin/openssl -> /usr/bin/openssl
lrwxrwxrwx 1 vsc40023 vsc40023 13 Mar  4 21:09 OpenSSL/1.1/lib64/libssl.so -> libssl.so.1.1

boegel
boegel approved these changes Mar 4, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit a1752d9 into easybuilders:develop Mar 4, 2022
@lexming lexming deleted the ssl-certs branch March 4, 2022 21:57
@bedroge bedroge mentioned this pull request May 11, 2022
Merged
@boegel boegel mentioned this pull request Jun 4, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

4.5.4

Development

Successfully merging this pull request may close these issues.

make OpenSSL aware of host CA certs OpenSSL installation does not refer to system certificate path

3 participants

@lexming @boegelbot @boegel