Cross-Database SQL for Live Databases, Files, and S3

Query multiple live sources in one SQL statement

Join MySQL, PostgreSQL, files, and S3-backed data from one workspace to validate, compare, and shape datasets before migration or automation.

The Problem

Why Cross-Database Queries Are Hard in Traditional Tools

Most traditional tools execute queries against a single active connection. Cross-connection joins typically require database-level federation or external pipelines.

Most IDEs Execute Against One Active Connection

DBeaver, DataGrip, and similar tools run queries against a single active connection per SQL editor tab. Cross-database joins depend on database-level federation such as FDW or linked servers.

Warehouses Centralize Data Before Joins

BigQuery, Snowflake, and Redshift typically require data to be ingested or staged before cross-source joins. External querying is possible but requires additional configuration and managed integrations.

ELT Platforms Focus on Moving Data First

Airbyte, Fivetran, and dbt focus on extracting and loading data into a destination before it can be joined or analyzed. Ad-hoc cross-source queries are not their primary workflow.

No Built-In Federated SQL Engine

Traditional database IDEs do not include a built-in federated SQL engine for ad-hoc joins across separate connections. Cross-connection joins typically require database-level configuration or manual export/import.

DBConvert Streams removes all of this overhead with a built-in federated SQL engine.

Architecture

Built-In Federated SQL Engine

An embedded DuckDB engine that attaches to your live sources and executes queries in place.

Attach

Each source is attached as a virtual schema

Plan

Query planned centrally by DuckDB optimizer

Execute

Results returned instantly from attached sources

No Staging

No persistent staging or replication required

Examples

Tested SQL Examples

Copy-paste queries across databases and files.

Note: pg1 and my1 are connection aliases defined in DBConvert Streams. They represent configured PostgreSQL and MySQL connections.

Cross-Database JOIN (PostgreSQL + MySQL)

SELECT
    a.first_name || ' ' || a.last_name AS actor_name,
    f.title AS film_title
FROM pg1.public.actor a
JOIN pg1.public.film_actor fa ON a.actor_id = fa.actor_id
JOIN my1.sakila.film f ON fa.film_id = f.film_id
LIMIT 20;

Joins tables across PostgreSQL and MySQL in a single query — no export, no import, no staging.

Cross-Database UNION (PostgreSQL + MySQL)

(SELECT 'PostgreSQL' as source, film_id, title, release_year
FROM pg1.public.film
WHERE title LIKE 'A%'
LIMIT 5)

UNION ALL

(SELECT 'MySQL' as source, film_id, title, release_year
FROM my1.sakila.film
WHERE title LIKE 'A%'
LIMIT 5);

Combines results from PostgreSQL and MySQL into one dataset.

Use Cases

When to Use Cross-Database SQL

Migration Validation

Compare data between source and target before cutover. Verify row counts, checksums, and content match across migration runs.

Pre-Migration Filtering

Use SQL to inspect and filter source data before starting a migration. Identify schema mismatches or data quality issues upfront.

Join Operational DB with S3 Export

Combine live database records with archived Parquet or CSV exports in S3 for historical analysis without re-importing.

Audit Across Environments

Investigate production vs staging mismatches. Run comparison queries directly without building replication streams first.

Cross-System Reporting

Combine operational data from MySQL and PostgreSQL into unified reports without a data warehouse.

File + Database Analysis

Analyze Parquet or CSV exports together with live databases using full DuckDB SQL - aggregations, window functions, and transformations.

Convert Mode

Your Query Becomes Your Pipeline

The SQL you write to explore is the same SQL that powers a stream. Point it at a target — another database, a file, S3 — and the data moves. No hand-off between tools. No rebuild.

  • Use SQL query results as a source for data migration in convert mode
  • Validate data before or after migration with cross-source comparison
  • Query file pipeline outputs (CSV, Parquet, JSON) alongside databases
  • Automate everything via REST API
SQL query results used as a stream source in convert mode configuration

Run the first query in the explorer

Use the explorer for hands-on query work and keep this page for patterns you can copy. Move into migration or API automation only after the SQL is doing the right job.