End-to-end data pipeline transforming Olist e-commerce data through Azure cloud services. Implements medallion architecture (Bronze-Silver-Gold) with multi-source ingestion, Spark-based processing, and OLTP-to-OLAP optimization for analytics-ready datasets.

A complete reference implementation of a local-first ecosystem for AI-powered analytics. This repository contains the source code for the SBDK.dev website, the central hub for the SBDK suite of open-source tools.

Standalone music intelligence layer that transforms your Spotify listening history into structured, filterable playlists with the goal of giving you explicit control over your taste.

A toml based data pipeline creation tool/framework

A full-stack financial application featuring real-time forex rates and machine learning-powered price predictions across multiple currency pairs. Built with automated data pipelines and Prophet time series forecasting.

This repository contains scripts to build and process a Real-Time GDP (RTD) dataset for Peru, focusing on extracting, cleaning, and analyzing GDP revisions from the BCRP's Weekly Reports. Future updates will include econometric models and visualizations.

An end-to-end, multi-container solution for collecting and visualizing stock market data

Production-grade cold email outreach infrastructure with strict data normalization, deterministic lead scoring, and fault-tolerant multi-stage campaigns.

End-to-End PV Monitoring & Streaming Pipeline with Delta Lake

Data automation involves automating the extraction, transformation, and loading (ETL) processes to streamline data workflows. GitHub Actions enables automated execution of tasks, such as building, testing, and deploying code, in response to events. This integration simplifies continuous deployment and ensures repeatable data pipeline operations

Template repo for setting object-oriented approach to creating a data pipeline

This project is an end-to-end machine learning pipeline for predicting diamond prices based on various features. It includes data preprocessing, model training, and prediction scripts, and is designed for easy setup and use by anyone familiar with Python and data science.

Event-driven, serverless data pipeline for securely syncing files from Google Drive to Microsoft SharePoint using chunked streaming and zero data retention.

Data-Engineering learning pg

An extensible, production-ready data pipeline framework for automatically fetching, transforming, and loading data from various APIs.

Automated data pipeline scraping fuel prices daily, apply transformations and load the data to a PostgreSQL database.

Automated pipeline to scrape and compare gaming gear prices & ratings from Amazon and eBay, including laptops, mice, Nintendo Switch, PS5, and Xbox. Outputs daily CSVs, SQL-ready datasets, and marketplace insights for competitive analysis.

“A modular DBT data warehouse pipeline built on SQL Server, featuring staging, intermediate, and mart layers with automated data quality tests and documentation.”

Multi-agent system for automated detection and analysis of failed Snowflake tasks using LLMs

Adobe ColdFusion-integrated email pipeline - Office 365 OAuth2/IMAP, PDF extraction, CSV export via cfexecute