This project demonstrates a streamlined ELT data pipeline, showcasing a seamless flow of data from a web application to advanced analytics. The pipeline integrates multiple Google Cloud Platform (GCP) services: the WebApp collects and ingests data, which is stored in Google Cloud Storage (GCS). A Cloud Function triggers automated transformations, loading the processed data into BigQuery for scalable, efficient analysis. Finally, Looker Studio visualizes insights, enabling interactive reporting and data-driven decision-making. This architecture highlights the end-to-end data journey, leveraging cloud-native services for a robust, scalable, and efficient solution.
- Global Retail Chain needs to consolidate and analyze its sales revenue
- The company needs a web application to load the sales data kept in csv from all the geogaphy to a single location
- From the single location the company wants to have analytics performed by it's Business Analyst
- A simple Web application to upload the csv files of sales data
- The Web Application should store these files in GCS Bucket
- When the file is uploaded to GCS bucket one GCP Cloud function event must be triggered
- The uploaded file in GCS should be loaded to Biggquery with help cloud run function
- Required transformation is performed in BigQuery to get the sales revenue values
- The transformed data must be saved in BigQuery View
- Using Looker Studio the Dashboardcan be created for the bigquery data








