This dbt package transforms data from Fivetran's Pendo connector into analytics-ready tables.

• Number of materialized models¹: 68
• Connector documentation
  • Pendo connector documentation
  • Pendo ERD
• dbt package documentation
  • GitHub repository
  • dbt Docs
  • DAG
  • Changelog
• dbt Core™ supported versions
  • >=1.3.0, <3.0.0

This package enables you to understand how users are experiencing and adopting your product. It creates enriched models with metrics focused on feature usage, page activity, and guide interactions.

Final output tables are generated in the following target schema:

<your_database>.<connector/schema_name>_pendo

By default, this package materializes the following final tables:

¹ Each Quickstart transformation job run materializes these models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as view, table, or incremental.

To use this dbt package, you must have the following:

• At least one Fivetran Pendo connection syncing data into your destination.
• A BigQuery, Snowflake, Redshift, PostgreSQL, or Databricks destination.

You can either add this dbt package in the Fivetran dashboard or import it into your dbt project:

• To add the package in the Fivetran dashboard, follow our Quickstart guide.
• To add the package to your dbt project, follow the setup instructions in the dbt package's README file to use this package.

Include the following pendo package version in your packages.yml file.

TIP: Check dbt Hub for the latest installation instructions or read the dbt docs for more information on installing packages.

# packages.yml
packages:
  - package: fivetran/pendo
    version: [">=1.3.0", "<1.4.0"] # we recommend using ranges to capture non-breaking changes automatically

All required sources and staging models are now bundled into this transformation package. Do not include fivetran/pendo_source in your packages.yml since this package has been deprecated.

If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your dbt_project.yml. This is required in order for the package to accurately search for macros within the dbt-labs/spark_utils then the dbt-labs/dbt_utils packages respectively.

dispatch:
  - macro_namespace: dbt_utils
    search_order: ['spark_utils', 'dbt_utils']

By default, this package runs using your destination and the pendo schema. If this is not where your Pendo data is (for example, if your Pendo schema is named pendo_fivetran), add the following configuration to your root dbt_project.yml file:

vars:
  pendo:
    pendo_database: your_database_name
    pendo_schema: your_schema_name

If you have multiple Pendo connections in Fivetran and would like to use this package on all of them simultaneously, we have provided functionality to do so. For each source table, the package will union all of the data together and pass the unioned table into the transformations. The source_relation column in each model indicates the origin of each record.

To use this functionality, you will need to set the pendo_sources variable in your root dbt_project.yml file:

# dbt_project.yml

vars:
  pendo:
    pendo_sources:
      - database: connection_1_destination_name # Required
        schema: connection_1_schema_name # Required
        name: connection_1_source_name # Required only if following the step in the following subsection

      - database: connection_2_destination_name
        schema: connection_2_schema_name
        name: connection_2_source_name

If you are running the package through Fivetran Transformations for dbt Core™, the below step is necessary in order to synchronize model runs with your Pendo connections. Alternatively, you may choose to run the package through Fivetran Quickstart, which would create separate sets of models for each Pendo source rather than one set of unioned models.

By default, this package defines one single-connection source, called pendo, which will be disabled if you are unioning multiple connections. This means that your DAG will not include your Pendo sources, though the package will run successfully.

To properly incorporate all of your Pendo connections into your project's DAG:

1. Define each of your sources in a .yml file in your project. Utilize the following template for the source-level configurations, and, most importantly, copy and paste the table and column-level definitions from the package's src_pendo.yml file.

# a .yml file in your root project

version: 2

sources:
  - name: <name> # ex: Should match name in pendo_sources
    schema: <schema_name>
    database: <database_name>
    loader: fivetran
    config:
      loaded_at_field: _fivetran_synced
      freshness: # feel free to adjust to your liking
        warn_after: {count: 72, period: hour}
        error_after: {count: 168, period: hour}

    tables: # copy and paste from pendo/models/staging/src_pendo.yml - see https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/ for how to use anchors to only do so once

2. Set the has_defined_sources variable (scoped to the pendo package) to True, like such:

# dbt_project.yml
vars:
  pendo:
    has_defined_sources: true

vars:
  pendo__feature_event_pass_through_columns: # will be passed to pendo__feature_event and stg_pendo__feature_event
    - name:           "custom_crazy_field_name"
      alias:          "normal_field_name"
  pendo__page_event_pass_through_columns: # will be passed to pendo__page_event and stg_pendo__page_event
    - name:           "property_field_id"
      alias:          "new_name_for_this_field_id"
      transform_sql:  "cast(new_name_for_this_field as int64)"
    - name:           "this_other_field"
      transform_sql:  "cast(this_other_field as string)"
  pendo__account_history_pass_through_columns: # will be passed to pendo__account, pendo__feature_event, and pendo__page_event
    - name:           "well_named_field_1"
  pendo__visitor_history_pass_through_columns: # will be passed to pendo__visitor, pendo__feature_event, and pendo__page_event 
    - name:           "well_named_field_2"
  pendo__event_pass_through_columns: # will be passed to stg_pendo__event only
    - name:           "well_named_field_3"

...
models:
    pendo:
      +schema: my_new_schema_name # Leave +schema: blank to use the default target_schema.
      staging:
        +schema: my_new_schema_name # Leave +schema: blank to use the default target_schema.

vars:
  pendo:
    pendo_<default_source_table_name>_identifier: your_table_name 

vars:
  pendo_group_identifier: '"Group"' # as an example, must include this quoting pattern and adjust for your exact casing

sources:
  tables:
    - name: group 
      # Add the below
      identifier: GROUP # Or what your group table is named, being mindful of casing
      quoting:
        identifier: true

This dbt package is dependent on the following dbt packages. These dependencies are installed by default within this package. For more information on the following packages, refer to the dbt hub site.

IMPORTANT: If you have any of these dependent packages in your own packages.yml file, we highly recommend that you remove them from your root packages.yml to avoid package version conflicts.

packages:
    - package: fivetran/fivetran_utils
      version: [">=0.4.0", "<0.5.0"]

    - package: dbt-labs/dbt_utils
      version: [">=1.0.0", "<2.0.0"]


    - package: dbt-labs/spark_utils
      version: [">=0.3.0", "<0.4.0"]

The Fivetran team maintaining this package only maintains the latest version of the package. We highly recommend you stay consistent with the latest version of the package and refer to the CHANGELOG and release notes for more information on changes across versions.

A small team of analytics engineers at Fivetran develops these dbt packages. However, the packages are made better by community contributions.

We highly encourage and welcome contributions to this package. Learn how to contribute to a package in dbt's Contributing to an external dbt package article.

• If you have questions or want to reach out for help, see the GitHub Issue section to find the right avenue of support for you.
• If you would like to provide feedback to the dbt package team at Fivetran or would like to request a new dbt package, fill out our Feedback Form.