tmctl dump - Add support for Docker-Compose & DigitalOcean by FranBarrera · Pull Request #197 · triggermesh/tmctl

FranBarrera · 2022-12-21T16:03:40Z

Added a new parameter to tmctl dump [--platform| -p] with multiple platforms [kubernetes | knative | docker-compose | digitalocean]. By default is kubernetes.

Example:

tmctl dump -p docker-compose

services:
  console:
    container_name: console
    environment: []
    image: gcr.io/triggermesh/triggermesh-console:v0.0.1
    ports:
    - 54772:8080
    volumes: []
  foo:
    command: start --memory.buffer-size 100 --memory.produce-timeout 1s --broker-config-path
      /etc/triggermesh/broker.conf
    container_name: foo
    environment: []
    image: gcr.io/triggermesh/memory-broker:latest
    ports:
    - 54758:8080
    volumes:
    - source: /Users/fran/.triggermesh/cli/foo/broker.conf
      target: /etc/triggermesh/broker.conf
      type: bind
  foo-httppollersource:
    container_name: foo-httppollersource
    environment:
    - HTTPPOLLER_EVENT_TYPE=buzzword.phrase
    - HTTPPOLLER_EVENT_SOURCE=local.foo-httppollersource
    - HTTPPOLLER_ENDPOINT=https://corporatebs-generator.sameerkumar.website/
    - HTTPPOLLER_METHOD=GET
    - HTTPPOLLER_SKIP_VERIFY=false
    - HTTPPOLLER_INTERVAL=20s
    - K_SINK=http://host.docker.internal:54758
    image: gcr.io/triggermesh/httppollersource-adapter:v1.22.0
    ports:
    - "8080"
    volumes: []
  foo-transformation:
    container_name: foo-transformation
    environment:
    - TRANSFORMATION_CONTEXT=[{"operation":"add","paths":[{"key":"type","value":"foo-transformation.output"}]}]
    - TRANSFORMATION_DATA=[{"operation":"add","paths":[{"key":"new-field","value":"hello
      from Transformation!"}]}]
    image: gcr.io/triggermesh/transformation-adapter:v1.22.0
    ports:
    - 54777:8080
    volumes: []

sebgoa · 2022-12-21T18:59:47Z

it works.

but for DO:

sources need to be of type workers and not services
the sources need to sink to the broker with a route of /broker like

  - key: K_SINK
    scope: RUN_AND_BUILD_TIME
    value: ${broker.PRIVATE_URL}/broker

the broker is indeed of type services but with a route set to broker like:

  name: broker
  routes:
  - path: /broker

sebgoa · 2022-12-21T21:40:21Z

Can you remove the gcr.io/triggermesh from the DO dump. Those images will need to be in the DO registry, right now we can reference them implicitly with only the adapter name.

For compose the full gcr.io/triggermesh name is correct.

cmd/dump/dump.go

pkg/triggermesh/components/service/service.go

pkg/triggermesh/components/source/source.go

pkg/triggermesh/constants.go

pkg/triggermesh/interfaces.go

FranBarrera · 2022-12-22T16:04:19Z

@sebgoa DO issues should be fine now

sebgoa · 2022-12-28T12:17:19Z

This could be a follow on ticket.

When there are more than one services (i.e targets) each should have its own route (instead of being just root /) and this should get reflected in the broker config if need be.

cmd/dump/dump.go

sebgoa · 2023-01-10T11:08:57Z

A bit dirty but I am putting a diff here so that I don't loose it

$ git diff cmd/dump/dump.go
diff --git a/cmd/dump/dump.go b/cmd/dump/dump.go
index fdaeba8..492f4ba 100644
--- a/cmd/dump/dump.go
+++ b/cmd/dump/dump.go
@@ -25,6 +25,7 @@ import (
        "strings"
 
        "github.com/digitalocean/godo"
+       "github.com/google/uuid"
        "github.com/spf13/cobra"
        "github.com/spf13/viper"
 
@@ -147,7 +148,7 @@ func (o *dumpOptions) dump() error {
                                                targetURL = fmt.Sprintf("http://%s:8080", target.Ref.Name)
                                        }
                                        if o.Platform == platformDigitalOcean {
-                                               targetURL = fmt.Sprintf("${%s.PRIVATE_URL}", target.Ref.Name)
+                                               targetURL = fmt.Sprintf("${%s.PRIVATE_URL}/%s", target.Ref.Name, target.Ref.Name)
                                        }
                                        trigger := tmbroker.LocalTriggerSpec{}
                                        filter := eventingbroker.Filter{}
@@ -214,7 +215,7 @@ func (o *dumpOptions) dump() error {
                        }
                }
                doObject := godo.AppSpec{
-                       Name:     "triggermesh",
+                       Name:     "triggermesh-" + uuid.NewString()[:5],
                        Services: doServices,
                        Workers:  doWorkers,
                        Region:   "fra",
(⎈ |prod:sebgoa)MacBook-Air:tmctl sebgoa$ git diff pkg/triggermesh/components/target/target.go
diff --git a/pkg/triggermesh/components/target/target.go b/pkg/triggermesh/components/target/target.go
index 20f6eb2..1407568 100644
--- a/pkg/triggermesh/components/target/target.go
+++ b/pkg/triggermesh/components/target/target.go
@@ -156,7 +156,7 @@ func (t *Target) AsDigitalOcean(additionalEnvs map[string]string) (*digitalocean
                HTTPPort: 8080,
                Routes: []*godo.AppRouteSpec{
                        {
-                               Path: "/",
+                               Path: "/" + t.Name,
                        },
                },
                Envs:

sebgoa

first functionality is there. so I am approving.

thanks for switching the transformation to a ServiceSpec

pkg/triggermesh/components/target/target.go

tzununbekov · 2023-01-18T07:53:07Z

Looks like I cannot merge my branch with this PR because it is based on the personal fork. @FranBarrera could you cherry-pick 3017928? Alternatively, we can merge this one and then mine on the top, but cherry-picking is preferable to keep the change history clean.

tmctl dump - Add support for Docker-Compose & DigitalOcean

2de3592

FranBarrera requested review from odacremolbap and tzununbekov December 21, 2022 16:03

FranBarrera added 2 commits December 21, 2022 17:12

Fix tests

0e2b264

Fix tests

380f313

tzununbekov suggested changes Dec 22, 2022

View reviewed changes

FranBarrera added 4 commits December 22, 2022 11:29

Fix some comments

a0ede54

revert change

d673594

Add missing file

827c335

Add DO services and workers

12ce3c6

Generate random port

d056428

FranBarrera requested a review from tzununbekov December 23, 2022 14:11

FranBarrera added 2 commits December 23, 2022 22:03

Fix envs

485f3ce

order

c80ce1d

sebgoa reviewed Jan 2, 2023

View reviewed changes

cmd/dump/dump.go Outdated Show resolved Hide resolved

jmcx linked an issue Jan 4, 2023 that may be closed by this pull request

Add a dump to docker-compose #54

Closed

FranBarrera added 2 commits January 11, 2023 11:55

Fix transformations

1c00b2c

Merge branch 'main' into compose-do

0aa93ce

FranBarrera requested a review from sebgoa January 11, 2023 10:56

sebgoa approved these changes Jan 11, 2023

View reviewed changes

pkg/triggermesh/components/target/target.go Outdated Show resolved Hide resolved

FranBarrera added 5 commits January 12, 2023 12:39

Fix conflicts

2457faa

Merge branch 'main' into compose-do

131fac7

Move broker entrypoint out

0c3106a

Add internalPort

196cfcb

Fix compose

a1fefe8

tzununbekov and others added 4 commits January 18, 2023 16:29

Initial support for multi-platform dump

2c7dbd3

Fix compose

9c132c8

Fix docs

5c18c44

Fix test

48a4714

tzununbekov approved these changes Jan 19, 2023

View reviewed changes

FranBarrera merged commit f805a28 into triggermesh:main Jan 19, 2023

FranBarrera deleted the compose-do branch January 19, 2023 09:01

Conversation

FranBarrera commented Dec 21, 2022

Uh oh!

sebgoa commented Dec 21, 2022

Uh oh!

sebgoa commented Dec 21, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FranBarrera commented Dec 22, 2022

Uh oh!

sebgoa commented Dec 28, 2022

Uh oh!

Uh oh!

sebgoa commented Jan 10, 2023

Uh oh!

sebgoa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tzununbekov commented Jan 18, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants