docker-py: skip broken ImageCollectionTest::test_pull_multiple

thaJeztah · thaJeztah · commit f2b25e498f8e · 2020-01-03T15:47:39.000+01:00
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).

After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.

However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.

Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:

    =================================== FAILURES ===================================
    ____________________ ImageCollectionTest.test_pull_multiple ____________________
    tests/integration/models_images_test.py:90: in test_pull_multiple
        assert len(images) == 1
    E   AssertionError: assert 2 == 1
    E    +  where 2 = len([&lt;Image: 'hello-world:linux'&gt;, &lt;Image: 'hello-world:latest'&gt;])

This patch temporarily skips the broken test until it is fixed upstream.

Signed-off-by: Sebastiaan van Stijn &lt;github@gone.nl&gt;
diff --git a/hack/make/test-docker-py b/hack/make/test-docker-py
@@ -10,9 +10,11 @@ source hack/make/.integration-test-helpers
 : "${DOCKER_PY_COMMIT:=4.1.0}"
 
 # custom options to pass py.test
+# TODO remove these skip once we update to a docker-py version that has https://github.com/docker/docker-py/pull/2485
 : "${PY_TEST_OPTIONS:=\
 --deselect=tests/integration/api_container_test.py::AttachContainerTest::test_attach_no_stream \
 --deselect=tests/integration/api_exec_test.py::ExecDemuxTest::test_exec_command_tty_stream_no_demux \
+--deselect=tests/integration/models_images_test.py::ImageCollectionTest::test_pull_multiple \
 --junitxml=${DEST}/junit-report.xml \
 }"
 (
