Support exec command by valaparthvi · Pull Request #6579 · redhat-developer/odo

valaparthvi · 2023-02-08T18:55:58Z

What type of PR is this:

What does this PR do / why we need it:

Which issue(s) this PR fixes:

Fixes #5701

PR acceptance criteria:

Unit test
Integration test
Documentation

How to test changes / Special notes to the reviewer:

git clone https://github.com/kadel/demo-odo-springboot.git
git checkout d5956f5
Copy the following content in the devfile.yaml

devfile.yaml

schemaVersion: 2.2.0
metadata:
  name: demo
variables:
  APP_IMAGE: quay.io/tkral/devbox-demo-app
  DEV_IMAGE: quay.io/tkral/devbox-demo-devbox

components:
  - container:
      endpoints:
        - name: 8080-tcp
          targetPort: 8080
      env:
        - name: DEBUG_PORT
          value: "5858"
      image: "{{DEV_IMAGE}}"
      mountSources: true
    name: tools
  - container:
      image: golang
    name: pi
  - name: dev-image
    image:
      imageName: "{{DEV_IMAGE}}"
      dockerfile:
        uri: ./devcontainer/Dockerfile
        buildContext: ${PROJECT_SOURCE}

  - name: app-image
    image:
      imageName: "{{APP_IMAGE}}"
      dockerfile:
        uri: ./src/main/docker/Dockerfile
        buildContext: ${PROJECT_SOURCE}

  - name: resource-app
    kubernetes:
      uri: config/default/app.yaml

  - name: resource-pvc
    kubernetes:
      uri: config/default/pvc.yaml

  - name: resource-service
    kubernetes:
      uri: config/default/service.yaml

  - name: resource-route
    kubernetes:
      uri: config/local/route.yaml

commands:
  - exec:
      commandLine: mvn package -Dmaven.test.skip=true
      component: tools
      group:
        isDefault: true
        kind: build
      workingDir: ${PROJECT_SOURCE}
    id: build

  - exec:
      commandLine: mvn spring-boot:run -Dspring-boot.run.profiles=postgres
      component: tools
      hotReloadCapable: true
      group:
        isDefault: true
        kind: run
      workingDir: ${PROJECT_SOURCE}
    id: run

  - exec:
      commandLine:
        java -Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=${DEBUG_PORT},suspend=n
        -jar target/*.jar
      component: tools
      group:
        isDefault: true
        kind: debug
      workingDir: ${PROJECT_SOURCE}
    id: debug

  - exec:
      commandLine: helm repo add bitnami https://charts.bitnami.com/bitnami && helm install my-db bitnami/postgresql
      component: tools
    id: deploy-db
  - exec:
      commandLine: go version
      component: pi
    id: deploy-pi-db
  - apply:
      component: resource-app
    id: deploy-app

  - apply:
      component: resource-pvc
    id: deploy-pvc

  - apply:
      component: resource-service
    id: deploy-service

  - apply:
      component: resource-route
    id: deploy-route

  - apply:
      component: app-image
    id: app-image

  - composite:
      commands:
        # - app-image
        # - deploy-pvc
        # - deploy-service
        # - deploy-route
        # - deploy-app
        - deploy-pi-db
        - deploy-db
      group:
        isDefault: true
        kind: deploy
    id: deploy

odo deploy

The first task(deploy-pi-db) should execute successfully, the second(deploy-db) should fail because of incorrect ServiceAccount setup. In both cases a log should be printed and the job should be deleted in 60s after its completion.
There is a third case where if the job continues to run after 60s minute, log/error will be printed after every 60s which will continue until the user interrupts. This can be tested by changing the image for container comp pi to geolandg or some non-existent image.

netlify · 2023-02-08T18:56:04Z

✅ Deploy Preview for odo-docusaurus-preview ready!

Name	Link
🔨 Latest commit	`c54a522`
🔍 Latest deploy log	https://app.netlify.com/sites/odo-docusaurus-preview/deploys/640094118b0f8a00083db4cd
😎 Deploy Preview	https://deploy-preview-6579--odo-docusaurus-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

odo-robot · 2023-02-08T19:02:01Z

OpenShift Unauthenticated Tests on commit fe15d56 finished successfully.
View logs: TXT HTML

odo-robot · 2023-02-08T19:02:41Z

Validate Tests on commit fe15d56 finished successfully.
View logs: TXT HTML

odo-robot · 2023-02-08T19:02:41Z

NoCluster Tests on commit fe15d56 finished successfully.
View logs: TXT HTML

odo-robot · 2023-02-08T19:03:17Z

Unit Tests on commit fe15d56 finished successfully.
View logs: TXT HTML

odo-robot · 2023-02-08T19:19:44Z

Kubernetes Tests on commit fe15d56 finished with errors.
View logs: TXT HTML

odo-robot · 2023-02-08T19:21:37Z

Windows Tests (OCP) on commit fe15d56 finished successfully.
View logs: TXT HTML

odo-robot · 2023-02-08T19:24:47Z

OpenShift Tests on commit fe15d56 finished successfully.
View logs: TXT HTML

odo-robot · 2023-02-08T19:29:53Z

Kubernetes Docs Tests on commit 3adf8e5 finished successfully.
View logs: TXT HTML

pkg/deploy/deploy.go

pkg/kclient/interface.go

rm3l

A few comments after testing it.

pkg/kclient/jobs.go

valaparthvi · 2023-02-16T13:40:01Z

after 1 minute, we advise the user to look at odo logs --deploy --follow
display the last 100 lines of the output if the execution fails to be consistent with odo deploy.

Signed-off-by: Parthvi Vala <[email protected]>

…re that Signed-off-by: Parthvi Vala <[email protected]>

…nto account Signed-off-by: Parthvi Vala <[email protected]>

…le pkg Signed-off-by: Parthvi Vala <[email protected]>

Signed-off-by: Parthvi Vala <[email protected]>

…unc doc and gofmt the files Signed-off-by: Parthvi Vala <[email protected]>

Signed-off-by: Parthvi Vala <[email protected]>

sonarqubecloud · 2023-03-02T12:19:18Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

No Coverage information
0.3% Duplication

feloy · 2023-03-02T14:29:03Z

When I execute a long-running command, and try to get the logs, as documented in the output of odo deploy, I don't get anything:

$ odo logs --deploy
no containers running in the specified mode for the component "my-nodejs-2"

I'm not sure if it is a problem on odo logs side, or if some labels are not set on the Job or on the Pod Template of the Job?

valaparthvi · 2023-03-02T16:31:37Z

When I execute a long-running command, and try to get the logs, as documented in the output of odo deploy, I don't get anything:
$ odo logs --deploy
no containers running in the specified mode for the component "my-nodejs-2"
I'm not sure if it is a problem on odo logs side, or if some labels are not set on the Job or on the Pod Template of the Job?

Are you running odo logs after odo deploy finishes or while it is running? If you run it before or after odo deploy is run, odo logs is bound to throw the no containers error. If that is not the case, can you share the devfile you're running?

feloy · 2023-03-02T16:44:44Z

When I execute a long-running command, and try to get the logs, as documented in the output of odo deploy, I don't get anything:
$ odo logs --deploy
no containers running in the specified mode for the component "my-nodejs-2"
I'm not sure if it is a problem on odo logs side, or if some labels are not set on the Job or on the Pod Template of the Job?
Are you running odo logs after odo deploy finishes or while it is running? If you run it before or after odo deploy is run, odo logs is bound to throw the no containers error. If that is not the case, can you share the devfile you're running?

I'm running odo logs while the job is running. I've used the devfile used for the integration test (devfile-deploy-exec.yaml), adding sleep 60 after the echo Hello world command:

commands:
  - exec:
      commandLine: npm install
      component: runtime
      group:
        isDefault: true
        kind: build
      workingDir: /project
    id: install
  - exec:
      commandLine: npm start
      component: runtime
      group:
        isDefault: true
        kind: run
      workingDir: /project
    id: run
  - exec:
      commandLine: npm run debug
      component: runtime
      group:
        isDefault: true
        kind: debug
      workingDir: /project
    id: debug
  - exec:
      commandLine: npm test
      component: runtime
      group:
        isDefault: true
        kind: test
      workingDir: /project
    id: test
  - exec:
      commandLine: echo Hello world; sleep 65; echo Bye
      component: runtime
    id: deploy-exec
  - id: deploy
    composite:
      commands:
        - deploy-exec
      group:
        kind: deploy
        isDefault: true
components:
  - container:
      endpoints:
        - name: http-3000
          targetPort: 3000
      image: registry.access.redhat.com/ubi8/nodejs-14:latest
      memoryLimit: 1024Mi
      mountSources: true
      sourceMapping: /project
    name: runtime
metadata:
  description: Stack with Node.js 14
  displayName: Node.js Runtime
  icon: https://nodejs.org/static/images/logos/nodejs-new-pantone-black.svg
  language: javascript
  name: nodejs-prj1-api-abhz
  projectType: nodejs
  tags:
    - NodeJS
    - Express
    - ubi8
  version: 1.0.1
schemaVersion: 2.2.0
starterProjects:
  - git:
      remotes:
        origin: https://github.com/odo-devfiles/nodejs-ex.git
    name: nodejs-starter
variables:
  CONTAINER_IMAGE: quay.io/unknown-account/myimage

valaparthvi · 2023-03-03T09:00:40Z

I cannot seem to reproduce this.

feloy · 2023-03-03T09:17:35Z

I cannot seem to reproduce this.

My bad, I cannot reproduce it either today.

feloy · 2023-03-03T09:18:40Z

Thanks @valaparthvi for this work

feloy · 2023-03-03T10:10:52Z

[FAILED] [33.509 seconds]
odo dev command tests
/go/odo_1/tests/integration/cmd_dev_test.go:34
  when a component is bootstrapped and pushed
  /go/odo_1/tests/integration/cmd_dev_test.go:64
    when a delay is necessary for the component to start and running odo dev
    /go/odo_1/tests/integration/cmd_dev_test.go:494
      [It] should first fail then succeed querying endpoint
      /go/odo_1/tests/integration/cmd_dev_test.go:512

Unrelated failing test
/override Kubernetes-Integration-Tests/Kubernetes-Integration-Tests

openshift-ci · 2023-03-03T10:10:56Z

@feloy: Overrode contexts on behalf of feloy: Kubernetes-Integration-Tests/Kubernetes-Integration-Tests

Details

In response to this:

[FAILED] [33.509 seconds]
odo dev command tests
/go/odo_1/tests/integration/cmd_dev_test.go:34
 when a component is bootstrapped and pushed
 /go/odo_1/tests/integration/cmd_dev_test.go:64
   when a delay is necessary for the component to start and running odo dev
   /go/odo_1/tests/integration/cmd_dev_test.go:494
     [It] should first fail then succeed querying endpoint
     /go/odo_1/tests/integration/cmd_dev_test.go:512

Unrelated failing test
/override Kubernetes-Integration-Tests/Kubernetes-Integration-Tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

* Support exec command for deploy Signed-off-by: Parthvi Vala <[email protected]> * Print log after timeout Signed-off-by: Parthvi Vala <[email protected]> * Add helper function to form proper commandLine Signed-off-by: Parthvi Vala <[email protected]> * Mockgen kclient Signed-off-by: Parthvi Vala <[email protected]> * Enhance error message Signed-off-by: Parthvi Vala <[email protected]> * Attempt at fixing unit test failures Signed-off-by: Parthvi Vala <[email protected]> * Rename import v1 to batchv1 Signed-off-by: Parthvi Vala <[email protected]> * Remove TODOs Signed-off-by: Parthvi Vala <[email protected]> * Add integration tests and cleanup on user interrupt Signed-off-by: Parthvi Vala <[email protected]> * Temp changes Signed-off-by: Parthvi Vala <[email protected]> * Log tip to run odo logs after a minute Signed-off-by: Parthvi Vala <[email protected]> * List components to delete even if there are no devfile resources Signed-off-by: Parthvi Vala <[email protected]> * Fix integration tests Signed-off-by: Parthvi Vala <[email protected]> * Fix deploy exec delete integration test Signed-off-by: Parthvi Vala <[email protected]> * Temp Change * Fix delete command tests * Fix mockgen client * Fix validation errors Signed-off-by: Parthvi Vala <[email protected]> * Fix unit test failure Signed-off-by: Parthvi Vala <[email protected]> * Attemp at writing less flaky integration test Signed-off-by: Parthvi Vala <[email protected]> * Remove TODOs Signed-off-by: Parthvi Vala <[email protected]> * Add tip after 1 minute and return the go routine if job finishes before that Signed-off-by: Parthvi Vala <[email protected]> * Use the container as it is so that container-overrides can be taken into account Signed-off-by: Parthvi Vala <[email protected]> * Move job spec code to a different helper function inside the libdevfile pkg Signed-off-by: Parthvi Vala <[email protected]> * Modify the Execute method to use the new helper function and refactoring Signed-off-by: Parthvi Vala <[email protected]> * Attempt at fixing integration and unit tests Signed-off-by: Parthvi Vala <[email protected]> * Move defer to print remaining resources to a separate function, fix func doc and gofmt the files Signed-off-by: Parthvi Vala <[email protected]> * Fix test failures Signed-off-by: Parthvi Vala <[email protected]> * Cleanup Signed-off-by: Parthvi Vala <[email protected]> * Cleanup unused functions Signed-off-by: Parthvi Vala <[email protected]> --------- Signed-off-by: Parthvi Vala <[email protected]>

openshift-ci bot requested review from anandrkskd and rm3l February 8, 2023 18:56

valaparthvi changed the title ~~Deploy exec~~ Support exec command Feb 8, 2023

rm3l reviewed Feb 13, 2023

View reviewed changes

pkg/deploy/deploy.go Outdated Show resolved Hide resolved

rm3l reviewed Feb 13, 2023

View reviewed changes

pkg/deploy/deploy.go Outdated Show resolved Hide resolved

rm3l reviewed Feb 13, 2023

View reviewed changes

pkg/kclient/interface.go Outdated Show resolved Hide resolved

rm3l reviewed Feb 13, 2023

View reviewed changes

pkg/kclient/jobs.go Show resolved Hide resolved

valaparthvi force-pushed the deploy-exec branch from 9d59bf5 to f48d25e Compare February 16, 2023 17:53

valaparthvi changed the title ~~Support exec command~~ WIP: Support exec command Feb 16, 2023

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Feb 16, 2023

valaparthvi force-pushed the deploy-exec branch from 8c2fd60 to 65c38a1 Compare February 17, 2023 08:06

valaparthvi closed this Feb 17, 2023

valaparthvi reopened this Feb 17, 2023

valaparthvi changed the title ~~WIP: Support exec command~~ Support exec command Feb 17, 2023

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Feb 17, 2023

valaparthvi changed the title ~~Support exec command~~ WIP: Support exec command Feb 17, 2023

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Feb 17, 2023

valaparthvi changed the title ~~WIP: Support exec command~~ Support exec command Feb 17, 2023

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Feb 17, 2023

valaparthvi added 14 commits March 2, 2023 17:48

Fix mockgen client

ef8e083

Fix validation errors

d9ea411

Signed-off-by: Parthvi Vala <[email protected]>

Fix unit test failure

3e4eecc

Signed-off-by: Parthvi Vala <[email protected]>

Attemp at writing less flaky integration test

caadaa5

Signed-off-by: Parthvi Vala <[email protected]>

Remove TODOs

a3c8048

Signed-off-by: Parthvi Vala <[email protected]>

Add tip after 1 minute and return the go routine if job finishes befo…

8f3accc

…re that Signed-off-by: Parthvi Vala <[email protected]>

Use the container as it is so that container-overrides can be taken i…

855d79e

…nto account Signed-off-by: Parthvi Vala <[email protected]>

Move job spec code to a different helper function inside the libdevfi…

04834e9

…le pkg Signed-off-by: Parthvi Vala <[email protected]>

Modify the Execute method to use the new helper function and refactoring

231d59d

Signed-off-by: Parthvi Vala <[email protected]>

Attempt at fixing integration and unit tests

90dee37

Signed-off-by: Parthvi Vala <[email protected]>

Move defer to print remaining resources to a separate function, fix f…

f4c2a92

…unc doc and gofmt the files Signed-off-by: Parthvi Vala <[email protected]>

Fix test failures

857a0db

Signed-off-by: Parthvi Vala <[email protected]>

Cleanup

e60f841

Signed-off-by: Parthvi Vala <[email protected]>

Cleanup unused functions

c54a522

Signed-off-by: Parthvi Vala <[email protected]>

valaparthvi force-pushed the deploy-exec branch from 7e9add1 to c54a522 Compare March 2, 2023 12:18

valaparthvi changed the title ~~WIP: Support exec command~~ Support exec command Mar 2, 2023

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Mar 2, 2023

feloy approved these changes Mar 3, 2023

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Mar 3, 2023

openshift-merge-robot merged commit 775adfd into redhat-developer:main Mar 3, 2023

Conversation

valaparthvi commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for odo-docusaurus-preview ready!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odo-robot bot commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rm3l left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

valaparthvi commented Feb 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sonarqubecloud bot commented Mar 2, 2023

Uh oh!

feloy commented Mar 2, 2023

Uh oh!

valaparthvi commented Mar 2, 2023

Uh oh!

feloy commented Mar 2, 2023

Uh oh!

valaparthvi commented Mar 3, 2023

Uh oh!

feloy commented Mar 3, 2023

Uh oh!

feloy commented Mar 3, 2023

Uh oh!

feloy commented Mar 3, 2023

Uh oh!

openshift-ci bot commented Mar 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

valaparthvi commented Feb 8, 2023 •

edited

Loading

netlify bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

odo-robot bot commented Feb 8, 2023 •

edited

Loading

valaparthvi commented Feb 16, 2023 •

edited

Loading