Skip to content

Cobra completion v2 with CLI plugin support#3429

Merged
thaJeztah merged 1 commit intodocker:masterfrom
ndeloof:cobra_completion_v2
May 12, 2022
Merged

Cobra completion v2 with CLI plugin support#3429
thaJeztah merged 1 commit intodocker:masterfrom
ndeloof:cobra_completion_v2

Conversation

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Feb 17, 2022

- What I did
Updated Cobra to 1.3.0 and started adopting Completion v2 mechanism with dynamic flag completion

Longer terms plan is to be able to fully generate completion for all commands, and replace the hand-written completion script

- How I did it
Enabled completion command and introduced registerCompletionFunc..

I would prefer this is all set by SetupRootCommand, but we need dockerCli to list available values, and passing this will change method signature. Or we need to set a global util.* variable like kubernetes does

Root Command gets a custom ValidArgsFunction set so that we can list available CLI plugins and include them in the completions. Note: they appear after canonical commands, unsorted:

$ docker c [tab] [tab]
checkpoint  commit      compose     config      container   context     cp          create 
...
 ~ docker compose - [tab] [tab]
--ansi               (Control when to print ANSI control characters ("never"|"always"|"auto"))
--compatibility      (Run compose in backward compatibility mode)
--env-file           (Specify an alternate environment file.)
...

Note: support by plugin require docker/cli dependency to be updated and Metadata to explicitly declare Completion support (SchemaVersion 0.2.0)

- How to verify it
generate completion script by docker completion bash > /usr/local/etc/bash_completion.d/docker.sh and check completion within a new console

or directly invoke completion by:

$ docker __complete --context "de"
desktop-linux
default
:0
Completion ended with directive: ShellCompDirectiveDefault

demo: https://asciinema.org/a/cThdD50ILb3G5Ugx9q25XlTGo

- Description for the changelog
Introduce generation of shell completion command docker completion with dynamic completion

- A picture of a cute animal (not mandatory but encouraged)

@ndeloof ndeloof force-pushed the cobra_completion_v2 branch 2 times, most recently from 4b68b6b to df0647c Compare February 17, 2022 10:23
@ndeloof ndeloof changed the title skeleton to adopt Cobra completion v2 Cobra completion v2 Feb 17, 2022
@ndeloof ndeloof force-pushed the cobra_completion_v2 branch from df0647c to 3be87bb Compare February 17, 2022 14:06
@codecov-commenter
Copy link

codecov-commenter commented Feb 17, 2022

Codecov Report

Merging #3429 (19ef618) into master (bfa9b40) will decrease coverage by 0.40%.
The diff coverage is 11.63%.

❗ Current head 19ef618 differs from pull request most recent head 003dc15. Consider uploading reports for the commit 003dc15 to get more accurate results

@@            Coverage Diff             @@
##           master    #3429      +/-   ##
==========================================
- Coverage   59.47%   59.06%   -0.41%     
==========================================
  Files         287      288       +1     
  Lines       24174    24372     +198     
==========================================
+ Hits        14377    14395      +18     
- Misses       8931     9105     +174     
- Partials      866      872       +6     

// Deprecated: experimental features are now always enabled in the CLI
Experimental bool `json:",omitempty"`
// Completions specifies whether the plugin supports command completion
Completions bool `json:",omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should consider updating the SchemaVersion as well

// SchemaVersion describes the version of this struct. Mandatory, must be "0.1.0"

Do you think we can combine this PR with #3254 ? (feel free to take that branch of course, if there's useful bits in it)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about #3254. I understand we want to keep backward compatibility for users, but on the other hand using cobra completion v2 bring more flexibility and CLI plugins support, so should be preferred. If we support both, we will have to offer double maintenance...

return nil, cobra.ShellCompDirectiveError
}

return filterForCompletion(names, toComplete), cobra.ShellCompDirectiveDefault

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are no context that match what the user has typed, do you want the shell to suggest file name?
If not, you should return cobra.ShellCompDirectiveNoFileComp. For example:

$ docker --context D[tab][tab]  # There is a context starting with 'D'
Development
$ docker --context Do[tab][tab] # There is NO context starting with 'Do' so we would get file completion
Documents/   Dockerfile    Dockerfile.new   Dockerfile.ori

)
}

func filterForCompletion(values []string, toComplete string) []string {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to consider not filtering on toComplete but always returning all choices. Most implementations do the filtering, but with the latest version of Cobra, it is safe not to do it. The advantage is that it allows for fuzzy matching for some shells. In this case, fuzzy matching means that the shell can match what the user has typed not just as a prefix.

So let's say you have contexts test-dev, test-prod, test-local, and the user types prod, zsh and fish shells would be smart and match on test-prod. This improves the user experience. As for bash and powershell that don't support fuzzy matching, they will filter on prefix automatically.

If you want more details, you can have a look at how we did this for helm: helm/helm#10513

@ndeloof ndeloof force-pushed the cobra_completion_v2 branch 9 times, most recently from e7193c3 to 5a64693 Compare February 25, 2022 14:50
@ndeloof ndeloof changed the title Cobra completion v2 Cobra completion v2 with CLI plugin support Feb 25, 2022
ndeloof added a commit to ndeloof/compose that referenced this pull request Feb 25, 2022
@ndeloof ndeloof force-pushed the cobra_completion_v2 branch 3 times, most recently from 94d87b3 to 5a65670 Compare February 28, 2022 09:02
@ndeloof ndeloof force-pushed the cobra_completion_v2 branch 3 times, most recently from 09eeb1f to ae2b5da Compare March 15, 2022 16:46
@thaJeztah
Copy link
Member

I think this is a good start

Given that this provides an additional way to provide completion scripts (it's definitely not (yet) on par with the existing bash completion - although the other shells were less well-maintained, so perhaps for those it's "on-par"), I'm ok with including this, and then work on improving.

I did find some issues (details below), perhaps some of them are easy fixes (let me know if they are).

Here's what I tried (running within the dev container (make shell);

apk add --no-cache -qq bash-completion
. /etc/profile.d/bash_completion.sh
source <(docker completion bash)

Created some containers in some states;

make binary

docker create --name created-container busybox
docker run --name stopped-container busybox
docker run -dit --name running-container busybox
docker run -dit --name paused-container busybox
docker pause paused-container

Running docker rm shows all containers;

docker rm <TAB> <TAB>
created-container  paused-container   running-container  stopped-container

And docker stop shows all running containers;

docker stop <TAB> <TAB>
paused-container   running-container

But docker start also shows all containers (probably should show "stopped" containers);

docker start  <TAB> <TAB>
created-container  paused-container   running-container  stopped-container

Similar for docker pause and docker unpause;

docker pause <TAB> <TAB>
paused-container   running-container

docker unpause <TAB> <TAB>
paused-container   running-container

Tab completion for docker run flags looks ok (we have a lot of flags!);

docker run --
Display all 100 possibilities? (y or n)
--add-host               --cpu-rt-period          --device-write-iops      --health-interval        --kernel-memory          --name                   --quiet                  --tty
--attach                 --cpu-rt-runtime         --disable-content-trust  --health-retries         --label                  --network                --read-only              --ulimit
--blkio-weight           --cpu-shares             --dns                    --health-start-period    --label-file             --network-alias          --restart                --user
--blkio-weight-device    --cpus                   --dns-option             --health-timeout         --link                   --no-healthcheck         --rm                     --userns
--cap-add                --cpuset-cpus            --dns-search             --hostname               --link-local-ip          --oom-kill-disable       --runtime                --uts
--cap-drop               --cpuset-mems            --domainname             --init                   --log-driver             --oom-score-adj          --security-opt           --volume
--cgroup-parent          --detach                 --entrypoint             --interactive            --log-opt                --pid                    --shm-size               --volume-driver
--cgroupns               --detach-keys            --env                    --io-maxbandwidth        --mac-address            --pids-limit             --sig-proxy              --volumes-from
--cidfile                --device                 --env-file               --io-maxiops             --memory                 --platform               --stop-signal            --workdir
--cpu-count              --device-cgroup-rule     --expose                 --ip                     --memory-reservation     --privileged             --stop-timeout
--cpu-percent            --device-read-bps        --gpus                   --ip6                    --memory-swap            --publish                --storage-opt
--cpu-period             --device-read-iops       --group-add              --ipc                    --memory-swappiness      --publish-all            --sysctl
--cpu-quota              --device-write-bps       --health-cmd             --isolation              --mount                  --pull                   --tmpfs

Completion of flag values may need some improvement, for example, when trying if --network (or any other flag?) completed to a list of networks I could choose, it fell back to completing files;

docker run --network=<TAB>
.DS_Store               .golangci.yml           LICENSE                 VERSION                 demo/                   gometalinter.json.tmp   service/
.circleci/              .idea/                  MAINTAINERS             build/                  docker-bake.hcl         internal/               suggestions.patch
...

For a boolean flag (I picked --rm), things were slightly better; it didn't complete files and started to complete images;

docker run --rm <TAB>
<none>:<none>                                 docker:19.03-dind                             golang:latest                                 quay.io/centos/centos:stream8
...

However, it also completed to <none>:<none> (yes, those are a bit of a hack altogether); we should filter those out, or (alternatively), return them as digest

For commands that don't take an argument, such as docker volume ls or docker ps, we should prevent it from completing. e.g., trying the below resulted in completion looking at local files;

docker volume ls <TAB>
.DS_Store               .golangci.yml           LICENSE                 VERSION                 demo/                   gometalinter.json.tmp   service/
...

I guess we can take all commands that have cli.NoArgs, and for those set ShellCompDirectiveNoFileComp (I guess this would be the default for most/all commands and flags, with some exceptions only)

And the reverse docker build should only complete to directories, not files; looks like cobra defines a ShellCompDirectiveFilterDirs that allows this;

docker build
.DS_Store               .golangci.yml           LICENSE                 VERSION                 demo/                   gometalinter.json.tmp   service/
...
.gitignore              TESTING.md              contrib/                experimental/           scripts/                vendor.sum

}

// CompletionFn offers completion for swarm secrets
func CompletionFn(dockerCli command.Cli) command.CompletionFn {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing; so it looks like all completion functions (with the exception of image.CompletionFn() and container.CompletionFn()) are only used within the package itself;

Screenshot 2022-05-02 at 16 29 41

I generally like to keep things un-exported, unless needed; perhaps we should

  • un-export all of them (except for the image and container ones)
  • should we have more descriptive names (for the exported ones at least)? (e.g., CompleteNames() ?); mostly wondering if there will be more "specialized" completion functions (per my earlier comments w.r.t. "in specific states" etc), in which case a more descriptive name would help.

@ndeloof ndeloof force-pushed the cobra_completion_v2 branch 2 times, most recently from 574c1cb to b273ef1 Compare May 6, 2022 09:07
@ndeloof
Copy link
Contributor Author

ndeloof commented May 6, 2022

@thaJeztah I've addressed some of the reported completion issue, there's many places we can improve.
Typically docker run --network=<TAB> is easy to fix with

cmd.RegisterFlagCompletionFunc(
		"network",
		network.CompleteNames(dockerCli),
	)

but there's many other flags to address :)

@thaJeztah
Copy link
Member

I was about to give this one another look, but I see there's an issue in the latest iteration, @ndeloof

#0 3.704 + go build -o /out/docker-windows-386.exe -tags ' osusergo' -ldflags ' -w -X "github.com/docker/cli/cli/version.GitCommit=5ea118aab0" -X "github.com/docker/cli/cli/version.BuildTime=2022-05-06T09:40:21Z" -X "github.com/docker/cli/cli/version.Version=20.10.2-683-g5ea118aab0.m"' github.com/docker/cli/cmd/docker
#80 6.592 package github.com/docker/cli/cmd/docker
#80 6.592 	imports github.com/docker/cli/cli/command/commands
#80 6.592 	imports github.com/docker/cli/cli/command/checkpoint
#80 6.592 	imports github.com/docker/cli/cli/command/container
#80 6.592 	imports github.com/docker/cli/cli/command/network
#80 6.592 	imports github.com/docker/cli/cli/command/container: import cycle not allowed

@ndeloof
Copy link
Contributor Author

ndeloof commented May 9, 2022

moved completion functions to completion/functions.go to avoid import cycles between containers / networks packages

// CompleteContainerNames offers completion for container names and IDs
// By default, only names are returned.
// Set DOCKER_COMPLETION_SHOW_CONTAINER_IDS=yes to also complete IDs.
func CompleteContainerNames(dockerCli command.Cli, all bool, filters ...func(types.Container) bool) ValidArgsFn {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with keeping this for now, and iterate on this, but somewhat on the fence on the bool argument (which could also be a "filter" / "option") and doing the filtering client-side

I think most (if not all) of the filtering we need can be done with a --filter (although in some cases, we'd have to pass multiple filters). For example;

docker ps --filter status=exited --filter status=created
CONTAINER ID   IMAGE     COMMAND   CREATED          STATUS                      PORTS     NAMES
7b1647808d7b   busybox   "sh"      56 minutes ago   Exited (0) 56 minutes ago             stopped-container
2ee1ae95ecf4   busybox   "sh"      56 minutes ago   Created                               created-container

We could still look at implementing != as filter (--filter status!=running), but in most cases, we'd still have to be more specific (e.g. a paused container cannot be started, so only excluding running likely won't work)

@rumpl
Copy link
Member

rumpl commented May 11, 2022

Looks like all the comments were fixed, maybe we should merge this one?

@thaJeztah
Copy link
Member

LOL completion is funny; wondering if there's an option to prevent this (each <TAB> adds a new paused-container to the list; it probably completes automatically because there's only one match;

docker unpause paused-container paused-container paused-container paused-container paused-container paused-container paused-container paused-container

@thaJeztah
Copy link
Member

thaJeztah commented May 12, 2022

For the list of "improvements"; we should exclude default (built-in) networks;

docker network rm <TAB>
bridge                  docker_gwbridge         docker_mynetwork  host                    none

Also need to prevent docker network create from completing to local files

docker network create
# prints file list

And some tweaking for attach (probably shouldn't have the paused-container

docker container attach
cranky_varahamihira  paused-container     running-container

docker build now completes to directories only (nice!)

docker build
.circleci/         .idea/             cli-plugins/       demo/              e2e/               man/               scripts/           vendor/
.git/              build/             cmd/               dockerfiles/       experimental/      opts/              service/
.github/           cli/               contrib/           docs/              internal/          

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants