further reduce uses of libcontainer and update runc v1.0.0-rc93 by thaJeztah · Pull Request #4717 · containerd/containerd

thaJeztah · 2020-11-10T11:12:17Z

depends on:

pkg/cri/server: remove dependency on libcontainer/apparmor, libcontainer/utils #4715 pkg/cri/server: remove dependency on libcontainer/apparmor, libcontainer/utils
libcontainer: isolate libcontainer/devices opencontainers/runc#2679 libcontainer: isolate libcontainer/devices

After this, we only vendor:

libcontainer/user
- used in oci/spec_opts.go (perhaps this is something to move to runtime-spec?)
libcontainer/devices
- devices.HostDevices() (used in a single test in pkg/cri/server/container_create_linux_test.go)
- devices.DeviceFromPath(() (used in a single location; pkg/cri/opts/spec_linux.go)
libcontainer/nsenter ("fake" dependency; pulled in because it contains C code)

The updated version of runc and gocapability add support for new capabilities
added in kernel 5.9 (syndtr/gocapability@d983527...42c35b4)

CAP_PERFMON
CAP_BPF
CAP_CHECKPOINT_RESTORE

k8s-ci-robot · 2020-11-10T11:12:19Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

vendor.conf

theopenlab-ci · 2020-11-10T11:19:26Z

Build succeeded.

containerd-build-arm64 : FAILURE in 5m 46s (non-voting)

thaJeztah · 2020-11-10T11:22:53Z

Hmm.. something failing; perhaps it doesn't deal well with the fork?

fatal: reference is not a tree: c7f874992f5322b2783580edef6ba6ae3e4aaa6e
Error: Process completed with exit code 128.

thaJeztah · 2020-11-10T11:40:02Z

script/setup/install-runc

We need to check these scripts; I've been bit by similar approaches in docker/docker (given, those didn't use vendoring)

go get -d will use go modules to get the dependency, and therefore first get go modules from master, before switching to the selected commit, which sometimes may cause unexpected results (see moby/moby#41560)

theopenlab-ci · 2020-11-10T11:50:44Z

Build succeeded.

containerd-build-arm64 : FAILURE in 7m 29s (non-voting)

theopenlab-ci · 2020-11-10T20:01:44Z

Build succeeded.

containerd-build-arm64 : FAILURE in 5m 53s (non-voting)

thaJeztah · 2020-11-10T21:18:33Z

Interesting; are machined on GH Actions not cleaned up between builds?

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'fedora/32-cloud-base' version '32.20200422.0' is up to date...
==> default: Running provisioner: install-runc (shell)...
    default: Running: inline script
    default: + /root/go/src/github.com/containerd/containerd/script/setup/install-runc
    default: fatal: destination path '/root/go/src/github.com/opencontainers/runc' already exists and is not an empty directory.
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
Error: Process completed with exit code 1.

theopenlab-ci · 2020-11-18T22:47:14Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 6m 13s (non-voting)

theopenlab-ci · 2020-11-19T10:17:55Z

Build succeeded.

containerd-build-arm64 : FAILURE in 6m 02s (non-voting)

theopenlab-ci · 2020-11-19T11:53:39Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 6m 19s (non-voting)

thaJeztah · 2020-11-19T12:00:43Z

Interesting; https://github.com/containerd/containerd/pull/4717/checks?check_run_id=1423934789

 /usr/bin/install -c -m 644 libseccomp.pc '/usr/local/lib/pkgconfig'
+ ldconfig
+ rm -rf /tmp/tmp.QeIT0Nsktc
Cloning into '/home/runner/work/containerd/containerd/src/github.com/opencontainers/runc'...
fatal: protocol 'temporarily
https' is not supported
Error: Process completed with exit code 128.

"protocol 'temporarily https' is not supported" 🤔

edit: well, of course, I know 😂

It's attempting to parse vendor.conf, and uses this line to vendor a dependency:

# FIXME temporarily vendoring from my fork (https://github.com/opencontainers/runc/pull/2679)
github.com/opencontainers/runc                      isolate_device https://github.com/thaJeztah/runc.git

theopenlab-ci · 2020-11-19T12:19:39Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 5m 50s (non-voting)

thaJeztah · 2020-11-19T12:31:12Z

Hmmm... looks like something doesn't cleanup between runs

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'fedora/32-cloud-base' version '32.20200422.0' is up to date...
==> default: Running provisioner: install-runc (shell)...
    default: Running: inline script
    default: + /root/go/src/github.com/containerd/containerd/script/setup/install-runc
    default: fatal: destination path '/root/go/src/github.com/opencontainers/runc' already exists and is not an empty directory.
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
Error: Process completed with exit code 1.

AkihiroSuda · 2020-12-02T09:01:04Z

opencontainers/runc#2679 is now merged

theopenlab-ci · 2020-12-03T15:30:30Z

Build succeeded.

containerd-build-arm64 : FAILURE in 3m 28s (non-voting)

theopenlab-ci · 2020-12-03T17:36:43Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 5m 55s (non-voting)

theopenlab-ci · 2020-12-07T13:50:33Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 5m 51s (non-voting)

AkihiroSuda · 2021-01-27T05:43:49Z

Is this still draft?

Looks like this import was not needed for the test; simplified the test by just using the device-path (a counter would work, but for debugging, having the list of paths can be useful). Signed-off-by: Sebastiaan van Stijn <[email protected]>

full diff: opencontainers/runc@v1.0.0-rc92...v1.0.0-rc93 also removes dependency on libcontainer/configs Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah · 2021-02-04T15:18:05Z

After this; dependency on runc vendoring is reduced to;

$ tree vendor/github.com/opencontainers/runc/

vendor/github.com/opencontainers/runc/
├── LICENSE
├── NOTICE
└── libcontainer
    ├── devices
    │   ├── device.go
    │   ├── device_unix.go
    │   ├── device_windows.go
    │   └── devices.go
    └── user
        ├── MAINTAINERS
        ├── lookup.go
        ├── lookup_unix.go
        ├── lookup_windows.go
        └── user.go

3 directories, 11 files

theopenlab-ci · 2021-02-04T15:18:58Z

Build succeeded.

containerd-build-arm64 : NODE_FAILURE in 0s (non-voting)

estesp

LGTM

estesp · 2021-02-04T15:47:29Z

almost passed :( test timeout as it just didn't finish by 10m inside the Fedora-in-vagrant-on-macOS 🤓

thaJeztah · 2021-02-04T15:57:08Z

Does it need a longer timeout? Don't think we can restart only a single job, correct?

thaJeztah · 2021-02-04T16:43:24Z

/retest

dims · 2021-02-04T17:09:02Z

LGTM

the failed CI job seems be because of a bad download

2021-02-04T17:00:59.5843050Z     default: Downloading: https://vagrantcloud.com/fedora/boxes/33-cloud-base/versions/33.20201019.0/providers/virtualbox.box
2021-02-04T17:00:59.6362490Z 
2021-02-04T17:00:59.7170450Z �[KProgress: 0% (Rate: 0*/s, Estimated time remaining: --:--:--)
2021-02-04T17:00:59.7172020Z �[KDownload redirected to host: download.fedoraproject.org
2021-02-04T17:00:59.7708170Z 
2021-02-04T17:00:59.9694180Z �[KProgress: 100% (Rate: 1766/s, Estimated time remaining: --:--:--)
2021-02-04T17:01:00.0522640Z �[KProgress: 0% (Rate: 0/s, Estimated time remaining: --:--:--)
2021-02-04T17:01:00.0713810Z An error occurred while downloading the remote file. The error
2021-02-04T17:01:00.0714940Z message, if any, is reproduced below. Please fix this error and try
2021-02-04T17:01:00.0715840Z again.
2021-02-04T17:01:00.0716320Z 
2021-02-04T17:01:00.0716840Z The requested URL returned error: 404 Not Found

thaJeztah · 2021-02-04T17:17:49Z

ah, yes, the runc maintainers also ran into intermittent 404's for fedora; opencontainers/runc#2787

thaJeztah · 2021-02-04T17:24:15Z

opened #4999 to add the same hack

thaJeztah commented Nov 10, 2020

View reviewed changes

vendor.conf Outdated Show resolved Hide resolved

thaJeztah force-pushed the reduce_libcontainer_use branch 2 times, most recently from 010c9a4 to 64cca75 Compare November 10, 2020 11:35

thaJeztah commented Nov 10, 2020

View reviewed changes

thaJeztah force-pushed the reduce_libcontainer_use branch from 64cca75 to 050158b Compare November 10, 2020 11:41

thaJeztah force-pushed the reduce_libcontainer_use branch from 050158b to 0d4c476 Compare November 10, 2020 19:54

thaJeztah force-pushed the reduce_libcontainer_use branch from 0d4c476 to fb82710 Compare November 18, 2020 22:39

thaJeztah force-pushed the reduce_libcontainer_use branch from fb82710 to 9414ab7 Compare November 19, 2020 10:10

thaJeztah force-pushed the reduce_libcontainer_use branch from 9414ab7 to 0d0452b Compare November 19, 2020 11:45

thaJeztah force-pushed the reduce_libcontainer_use branch from 0d0452b to 3e6ce68 Compare November 19, 2020 12:12

thaJeztah force-pushed the reduce_libcontainer_use branch from 3e6ce68 to 666ca0f Compare December 3, 2020 15:25

thaJeztah force-pushed the reduce_libcontainer_use branch from 666ca0f to 417d802 Compare December 3, 2020 17:29

thaJeztah force-pushed the reduce_libcontainer_use branch from 417d802 to 57eb737 Compare December 7, 2020 13:43

thaJeztah mentioned this pull request Feb 4, 2021

update runc to v1.0.0-rc93 #4996

Closed

thaJeztah added 2 commits February 4, 2021 16:08

update runc to v1.0.0-rc93

04d061f

full diff: opencontainers/runc@v1.0.0-rc92...v1.0.0-rc93 also removes dependency on libcontainer/configs Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah force-pushed the reduce_libcontainer_use branch from 4be9cde to 04d061f Compare February 4, 2021 15:14

thaJeztah changed the title ~~[draft] further reduce uses of libcontainer~~ further reduce uses of libcontainer and update runc v1.0.0-rc93 Feb 4, 2021

thaJeztah marked this pull request as ready for review February 4, 2021 15:15

estesp approved these changes Feb 4, 2021

View reviewed changes

thaJeztah mentioned this pull request Feb 4, 2021

[draft] update to runc v1.0.0-rc92-215-gcf6c0741 #4678

Closed

AkihiroSuda approved these changes Feb 4, 2021

View reviewed changes

AkihiroSuda added this to the 1.5 milestone Feb 4, 2021

AkihiroSuda added impact/changelog cherry-pick/1.4.x labels Feb 4, 2021

AkihiroSuda mentioned this pull request Feb 4, 2021

vendor: update syndtr/gocapability #4919

Closed

thaJeztah mentioned this pull request Feb 4, 2021

ci/gha/fedora: retry vagrant up #4999

Merged

thaJeztah mentioned this pull request Feb 4, 2021

Prepare release notes for 1.5.0-beta.1 #5003

Merged

AkihiroSuda merged commit 5f2d02a into containerd:master Feb 5, 2021

thaJeztah deleted the reduce_libcontainer_use branch February 5, 2021 07:54

AkihiroSuda mentioned this pull request Feb 6, 2021

[master] kube-proxy doesn't start up due to "apply caps: operation not permitted" error kubernetes-sigs/kind#2058

Closed

zhsj mentioned this pull request Feb 8, 2021

[release/1.4] further reduce uses of libcontainer and update runc v1.0.0-rc93 containerd/cri#1617

Closed

thaJeztah mentioned this pull request Feb 15, 2021

Separate runc binary version from libcontainer version, and remove obsolete build-tags #5036

Merged

thaJeztah mentioned this pull request Mar 11, 2021

[release/1.4] update to runc binary to v1.0.0-rc93, and decouple version from vendor.conf #5166

Merged

thaJeztah mentioned this pull request Feb 8, 2023

Don't check for apparmor_parser to be present #5519

Merged

Conversation

thaJeztah commented Nov 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Nov 10, 2020

Uh oh!

Uh oh!

theopenlab-ci bot commented Nov 10, 2020

Uh oh!

thaJeztah commented Nov 10, 2020

Uh oh!

thaJeztah Nov 10, 2020

Choose a reason for hiding this comment

Uh oh!

theopenlab-ci bot commented Nov 10, 2020

Uh oh!

theopenlab-ci bot commented Nov 10, 2020

Uh oh!

thaJeztah commented Nov 10, 2020

Uh oh!

theopenlab-ci bot commented Nov 18, 2020

Uh oh!

theopenlab-ci bot commented Nov 19, 2020

Uh oh!

theopenlab-ci bot commented Nov 19, 2020

Uh oh!

thaJeztah commented Nov 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theopenlab-ci bot commented Nov 19, 2020

Uh oh!

thaJeztah commented Nov 19, 2020

Uh oh!

AkihiroSuda commented Dec 2, 2020

Uh oh!

theopenlab-ci bot commented Dec 3, 2020

Uh oh!

theopenlab-ci bot commented Dec 3, 2020

Uh oh!

theopenlab-ci bot commented Dec 7, 2020

Uh oh!

AkihiroSuda commented Jan 27, 2021

Uh oh!

thaJeztah commented Feb 4, 2021

Uh oh!

theopenlab-ci bot commented Feb 4, 2021

Uh oh!

estesp left a comment

Choose a reason for hiding this comment

Uh oh!

estesp commented Feb 4, 2021

Uh oh!

thaJeztah commented Feb 4, 2021

Uh oh!

thaJeztah commented Feb 4, 2021

Uh oh!

dims commented Feb 4, 2021

Uh oh!

thaJeztah commented Feb 4, 2021

Uh oh!

thaJeztah commented Feb 4, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

thaJeztah commented Nov 10, 2020 •

edited

Loading

thaJeztah commented Nov 19, 2020 •

edited

Loading