Proposal for ECS CLI v2

⚠️ DISCLAIMER: This document outlines our vision for ECS CLI v2. Not all of these commands or integrations will be in the release. We want to gather early feedback on the ideas listed in this issue to make sure we’re building something useful and pivot if necessary.

Amazon ECS released version 1 of the Amazon ECS CLI in 2015. The Amazon ECS CLI v1 simplifies the management of your Amazon ECS clusters, tasks, services, and ECR repositories by enabling you to create profiles and cluster configurations with default settings. While many customers have found the Amazon ECS CLI useful, we have received feedback that this level of abstraction does not go far enough for developers who must still interact directly with Amazon ECS primitives.

The Amazon ECS development team is in the planning stages for a new major version (v2) of the Amazon ECS CLI. We are planning to shift the focus of the Amazon ECS CLI from using Docker Compose to model tasks and services to helping you model your Amazon ECS applications in a more holistic manner. We view this as a shift towards application-first development with an emphasis on usability and developer productivity.

The following is our initial proposal, which we would like feedback on.

What is ECS CLI v2?

ECS CLI v2 is an opinionated view of application-first development. This new iteration will introduce the concepts of applications and projects. An application is an Amazon ECS service or task. An application can also encapsulate other AWS resources, such as an Amazon SQS queue or a Amazon DynamoDB table, to achieve a business capability. Applications are independently deployable. A project is a namespace for a collection of related applications. If you build your applications through the Amazon ECS CLI, you’ll get some AWS best practices by default and have a smooth workflow for taking your applications from development to production. The new version of the Amazon ECS CLI will integrate with Infrastructure as Code toolkits like the AWS Cloud Development Kit (CDK) and HashiCorp Terraform so that you can grow in complexity and scale beyond what the Amazon ECS CLI aims to provide by default.

Modeling, provisioning, and deploying applications are only part of the application lifecycle for the developer. The Amazon ECS CLI aims to support workflows around troubleshooting and debugging to help when things go wrong.

Domains

These are the proposed categories to help you navigate our top-level commands.

• Getting started: These commands get you started with an Amazon ECS project. You can choose one of our existing application templates, for example a load-balanced expressjs app, or your existing Dockerfile. You can also choose how to manage your infrastructure as code, using, for example, our custom manifest format, AWS CloudFormation, AWS Cloud Development Kit, or Terraform. These commands help you create your infrastructure as code in your repository.
  Commands: ecs init, ecs project init.
• Develop: After the project template is created, you can iterate on its architecture. For example, you might want to add or remove an application. Examples including adding a new load-balanced service or scheduled task, adding a database or secret to an existing application, or listing all your applications. To deploy these changes you can either use a pipeline to release your changes to multiple stages or do ad-hoc deployments.
  Commands: ecs apps add, ecs apps db, ecs apps deploy, ecs pipelines.
• Observe: After an application is running in your cluster, you can examine it for validation or to understand what is going on during an incident. These commands revolve around telemetry and logging.
  Commands: ecs logs, ecs health, ecs trace.
• Operate: Manage the lifecycle of your ECS resources. These commands are similar to the AWS CLI and are granular enough to let you manage your images, tasks, services, or container instances directly.
  Commands: ecs images, ecs tasks, ecs services.
• Test: Test your application code and your infrastructure. Validate that your application is going to work as expected before you deploy, debug your application by setting breakpoints, or perform stress tests to discover the pressure points of your system.
  Commands: ecs local, ecs debug.
• Cost: Get pricing information about Amazon ECS and a breakdown of your infrastructure costs.
  Commands: ecs pricing.

Commands
These top-level commands are proposal and may not be in the final release. We want to experiment and simplify our commands while we are in preview with your feedback. They are listed here to give everyone a list of use cases and the vision we’d like to support.

$ ecs --help
Commands:
  Getting started  
     init                                 Initialize a project

  Develop  
     project                              Manage your projects
     env                                  Manage your environments
     app                                  Manage your applications
     pipeline                             Manage your CodePipelines
       
  Observe 
     logs                                 Display logs
     health                               Display metrics
     trace                                Display x-ray trace summaries
     
  Operate
     cluster                             Manage your clusters
     task                                Manage your tasks
     service                             Manage your services
     image                               Manage your ECR images
     task-def                            Manage your task definitions
     
  Test
     local                                Run your ECS task definition locally.
     debug                                Live debug an existing application.
     
  Cost
     pricing                              Display a breakdown of the cost of your application

Options:
 -h, --help                               Output usage information
 -v, --version                            Output the version number
 
Examples:
- Display the help menu for `apps` command 
  $ ecs apps --help

Releases

We plan to start with use cases that are not available in other available command line interfaces. Therefore, we believe focusing on the Getting started and Develop command categories first would be the most beneficial to users who want to use the Amazon ECS CLI.

Our plan is to release these features in preview in the new aws/amazon-ecs-cli-v2 GitHub repository and gather feedback. You can also track our tasks under the project on GitHub. While still in preview you can expect breaking changes to the commands.

What’s happening to v1?

The v1 CLI will continue existing as is. So if it already satisfies your existing use-cases you don’t have to upgrade. You can have both the v1 CLI and v2 CLI in your automated scripts if you’d like. We’ll continue to support all v1 commands until the commands with similar functionality (marked with ➡️ or 🔀) are ported over to v2. This includes bug-fixes, security patches and incremental updates as new ECS features are launched. However, once ported we will stop supporting the 🚫 dropped commands.

• ➡️ Available: We will port the command’s existing functionality over but the name of the command might be different (e.g. push ➡ images push). We will continue to do bug-fixes, security patches, and incremental ECS feature updates until we have the equivalent functionality in v2.
• 🔀 Available with caveats: The functionality will exist in v2, however you’ll have a different workflow. You might instead have to modify your infrastructure as code and do a deployment, or the action will be part of another command. We will continue to do bug-fixes, security patches, and incremental ECS feature updates until we have the including workflow in v2.
• 🚫 Dropped: For these commands, you can expect no more active development. That means no bug fixes, security patches, or new features.

v1 to v2 migration path

We’re planning on adding a ecs-cli migrate cluster command to v1 to move the ECS cluster and VPC created with ecs-cli up under a project. Under the hood, this command will add a tag to these resources so we can group them under the same namespace. This way a new application created with the v2 CLI can leverage the same cluster and VPC. It will also enable you to use the --project flag for any command that supports it. For example, you should be able to use ecs clusters scale --project myProject to change the number of container instances.

If you are a heavy user of ecs-cli compose <service> create, we are planning on adding a --file flag. This way you can transform your Compose file and ecs-params.yml to create a task-definition and optionally a service-definition JSON file. You can then use the operational commands such as ecs tasks ors ecs task-defs similar to the AWS CLI to push these resources.
If you use ecs-cli compose part of your CD scripts and want to migrate to v2, our recommendation is for you to add the generated task definition or service file to your repository, modify those instead, and then use the operation commands to deploy your changes.

Feedback

We would like to hear your thoughts on this draft direction for v2 of the CLI. Please let us know in the comments below. Specifically, we’d like to know:

• Are there any use-cases that we’re missing that you’d like to see part of the v2 CLI?
• Are you dependent on v1’s Docker Compose functionality to create ECS tasks and services? If so why?

This feedback is subject to the AWS Customer Agreement, particularly Section 8.5 (Suggestions).