Docker Cloud Deployment

This guide covers deploying DBConvert Streams to cloud hosting providers such as AWS, Google Cloud, Azure, DigitalOcean, Hetzner.

Video Walkthrough

For a comprehensive step-by-step video walkthrough of the complete deployment process, watch the following tutorial:

Watch on YouTube

This video covers server setup, one-line installation, UI launch, and API activation.

Quick Start

Use the one-line installer from the Installation section below.

DigitalOcean Marketplace

For the easiest cloud deployment option, DBConvert Streams is available as a 1-Click App on the DigitalOcean Marketplace: DBConvert Streams on DigitalOcean Marketplace

For prerequisites and deployment modes, see Docker Deployment Overview. You also need a cloud server instance with a public IP address.

Required Ports

When deploying to a cloud hosting provider, you need to ensure the following ports are open in your hosting provider's security group or firewall:

Port	Service	Description
22	SSH	For SSH access to the server
8080	HTTP	For Streams UI and API access
443	HTTPS	For secure UI and API access (if using HTTPS)

These ports must be opened on your hosting provider to allow incoming connections from the internet. All other services run internally and don't need exposed ports.

The API is accessible through Nginx at http://HOST_ADDR:8080/api/ (HTTP) or https://HOST_ADDR/api/ (HTTPS).

Installation

Basic Install

Using `curl`:

curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Using `wget`:

wget -qO- https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

This automatically:

installs Docker Compose if needed
sets up networking and shared volumes
detects the public or local IP
configures Nginx
prepares SSL certificate support

Advanced Install Options

Flag	Description
`-v, --version VERSION`	Install a specific version
`-d, --directory DIR`	Custom install directory
`-s, --secure`	Generate SSL certificates during install

Example:

# Install specific version to custom directory with HTTPS
./install.sh -v 2.0.0 -d /opt/dbconvert-streams -s

IP Address Configuration

DBConvert Streams automatically detects and configures your server's public IP address for proper SSL certificate generation and network settings.

If you're experiencing connection issues, you can manually specify the IP address:

./start.sh --ip YOUR_PUBLIC_IP

Cloud Provider-Specific Instructions

DBConvert Streams is not tied to any specific cloud vendor. You can install it on any hosting provider or VM environment that meets the Docker and networking requirements.

Use the generic install flow above on any supported VM. For vendor-specific setup details, use the dedicated guides:

Managing Services

Start Services

Before starting the services, obtain your API key from the Account page.

# Start with standard HTTP
./start.sh
# Start with HTTPS (requires generated certificates)
./start.sh --secure
# Explicitly use HTTP only
./start.sh --no-secure

If you need to manually specify the IP address:

./start.sh --ip YOUR_PUBLIC_IP

Stop Services

To temporarily stop all services while preserving data:

./stop.sh

Update Services

To update the DBConvert Streams services to a newer version:

# Update to latest version
./update.sh

# Update to specific version
./update.sh --version v2.0.0

Service Architecture

See Docker Deployment Overview — Service architecture for the full service list and deployment modes.

Security Considerations

Firewall Configuration

Each cloud provider has its own firewall/security group system. You'll need to allow these ports:

Port 22 (SSH)
Port 8080 (HTTP)
Port 443 (HTTPS) - if using HTTPS

The installation automatically sets up Nginx as a reverse proxy with self-signed SSL certificates. While these work for testing, they cause browser warnings in production. For a production environment:

Option 1: Use Let's Encrypt (Recommended)

Register a domain and point it to your server's IP address

Install Certbot:

sudo apt update
sudo apt install -y certbot python3-certbot-nginx

Obtain certificates:
```
sudo certbot --nginx -d yourdomain.com
```
Start with HTTPS enabled:
```
./start.sh --secure
```

Certbot automatically configures Nginx to use the new certificates, so no additional configuration is needed.

Option 2: Use Your Own Certificates

If you already have SSL certificates from another provider:

Place your certificates in the config/certs directory:
- config/certs/certificate.crt (your certificate file)
- config/certs/private.key (your private key file)
Start with HTTPS enabled:
```
./start.sh --secure
```

SSH Tunneling

For secure remote access to services, you can use SSH tunneling. This is especially useful when you want to access services through Nginx's reverse proxy:

# Forward local port 8080 to remote Nginx HTTP port
ssh -L 8080:localhost:8080 root@YOUR_SERVER_IP

After establishing the tunnel, access the UI at http://localhost:8080.

This provides a secure way to access your services through an encrypted SSH tunnel.

Troubleshooting

Common Issues & Fixes

Issue	Fix
Docker not running?	`sudo systemctl start docker`
Docker Compose not found?	`docker compose version`
Wrong IP detected?	`./start.sh --ip YOUR_IP`
HTTPS not working?	`./generate_certs.sh` → `./start.sh`

Check Service Status

docker compose ps

View logs:

docker compose logs -f [service_name]

Available services for log viewing:

stream-ui - Web interface
stream-api - REST API service
stream-reader - Database reader service
stream-writer-1, stream-writer-2 - Database writer services
nginx - Reverse proxy and SSL termination
nats - Message broker
vault - Secrets management (corporate mode)
consul - Service discovery (corporate mode)

Example:

# View API service logs
docker compose logs -f stream-api

# View writer logs
docker compose logs -f stream-writer-1

Find Vault Token (corporate mode)

If running in corporate mode and you need the Vault token:

grep VAULT_TOKEN install.log

The output will show your token in the format: VAULT_TOKEN=hvs.xxxxxxxxxxxxxxxx

IP Address Issues

If you're experiencing connection issues related to the IP address:

Check what IP address is being used:

cat config/host_info.conf

You'll see information like:

PUBLIC_IP="34.172.226.34"
ENVIRONMENT="Cloud (gcp)"
DETECTED_AT="2025-04-07 16:30:34 UTC"

If the IP is incorrect, specify the correct IP manually:
```
./start.sh --ip YOUR_PUBLIC_IP
```

Uninstallation

To completely remove DBConvert Streams:

./uninstall.sh

This will:

Stop and remove all containers
Delete the Docker network
Remove logs and configs
Prompt before deleting data