Skip to content

TAK-NZ/etl-poweroutages

Repository files navigation

ETL-PowerOutages

NZ Power Outage Monitoring

Data Source

Power Outages API - Aggregated power outage data from New Zealand electricity distribution boards including Orion Group, PowerCo, and Wellington Electricity.

Example Data

Power outages are displayed on the TAK map with the following information:

  • Utility name and affected area
  • Number of customers affected
  • Outage cause and status
  • Estimated restoration time
  • Crew status (when available)
  • Affected streets and areas

Power Outages on TAK Map

Deployment

Deployment into the CloudTAK environment for ETL tasks is done via automatic releases to the TAK.NZ AWS environment.

Github actions will build and push docker releases on every version tag which can then be automatically configured via the CloudTAK API.

GitHub Actions Setup

The workflow uses GitHub variables and secrets to make it reusable across different ETL repositories.

Organization Variables (recommended)

  • DEMO_STACK_NAME: Name of the demo stack (default: "Demo")
  • PROD_STACK_NAME: Name of the production stack (default: "Prod")

Organization Secrets (recommended)

  • DEMO_AWS_ACCOUNT_ID: AWS account ID for demo environment
  • DEMO_AWS_REGION: AWS region for demo environment
  • DEMO_AWS_ROLE_ARN: IAM role ARN for demo environment
  • PROD_AWS_ACCOUNT_ID: AWS account ID for production environment
  • PROD_AWS_REGION: AWS region for production environment
  • PROD_AWS_ROLE_ARN: IAM role ARN for production environment

Repository Variables

  • ETL_NAME: Name of the ETL (default: repository name)

Repository Secrets (alternative to organization secrets)

  • AWS_ACCOUNT_ID: AWS account ID for the environment
  • AWS_REGION: AWS region for the environment
  • AWS_ROLE_ARN: IAM role ARN for the environment

These variables and secrets can be set in the GitHub organization or repository settings under Settings > Secrets and variables.

Manual Deployment

For manual deployment you can use the scripts/etl/deploy-etl.sh script from the CloudTAK repo. As an example:

../CloudTAK/scripts/etl/deploy-etl.sh Demo v1.0.0 --profile tak-nz-demo

CloudTAK Configuration

When registering this ETL as a task in CloudTAK:

  • Use the <repo-name>.png file in the main folder of this repository as the Task Logo
  • Use the raw GitHub URL of this README.md file as the Task Markdown Readme URL

This will ensure proper visual identification and documentation for the task in the CloudTAK interface.

Configuration Options

  • API_URL: Power Outages API endpoint (default: https://utils.tak.nz/power-outages/outages)
  • Min Customers: Minimum number of customers affected to display (default: 0)
  • Utility Filter: Filter by specific utility (e.g., ORION_NZ, POWERCO_NZ, WELLINGTON_NZ)
  • Outage Type: Filter by outage type (planned, unplanned)

Recommended Styling Overrides

Color code outages by customer impact using CloudTAK styling overrides:

White (Minor) - 1-9 customers

  • Color: #FFFFFF or rgb(255, 255, 255)
  • JSONata: properties.metadata.customersAffected >= 1 and properties.metadata.customersAffected < 10

Yellow (Small) - 10-49 customers

  • Color: #FFFF00 or rgb(255, 255, 0)
  • JSONata: properties.metadata.customersAffected >= 10 and properties.metadata.customersAffected < 50

Orange (Medium) - 50-99 customers

  • Color: #FFA500 or rgb(255, 165, 0)
  • JSONata: properties.metadata.customersAffected >= 50 and properties.metadata.customersAffected < 100

Red (Large) - 100-499 customers

  • Color: #FF0000 or rgb(255, 0, 0)
  • JSONata: properties.metadata.customersAffected >= 100 and properties.metadata.customersAffected < 500

Purple (Major) - 500+ customers

  • Color: #800080 or rgb(128, 0, 128)
  • JSONata: properties.metadata.customersAffected >= 500

Development

TAK.NZ provided Lambda ETLs are currently all written in NodeJS through the use of a AWS Lambda optimized Docker container. Documentation for the Dockerfile can be found in the AWS Help Center

npm install

Add a .env file in the root directory that gives the ETL script the necessary variables to communicate with a local ETL server. When the ETL is deployed the ETL_API and ETL_LAYER variables will be provided by the Lambda Environment

{
    "ETL_API": "http://localhost:5001",
    "ETL_LAYER": "19"
}

To run the task, ensure the local CloudTAK server is running and then run with typescript runtime or build to JS and run natively with node

ts-node task.ts
npm run build
cp .env dist/
node dist/task.js

License

TAK.NZ is distributed under AGPL-3.0-only Copyright (C) 2025 - Christian Elsen, Team Awareness Kit New Zealand (TAK.NZ)

About

TAK ETL: Power Outage data from some NZ utility companies

Topics

Resources

License

Stars

Watchers

Forks

Contributors