PRC

• 

  Data dashboard - showing CPU/memory usage and wait stats (streamlit)

Inspiration

Our dashboard (i.e. frontend of the solution) is inspired by modern cloud infrastructure's monitoring dashboard (Google Cloud & Azure), the DataOps workflow's inspired by Openmesh roadmap.

What it does

A cloud data solution on AWS instance that load real-time WebSocket data (stock market) into the database, provide a frontend to show and monitor the data, including the connection string availability for further BI use.

How we built it

Back-end

ETL process and data of stock market via WebSocket (finnhub), utilizing python' lib pyodbc , json, pandas, websocket

pip install websocket pyodbc pandas json

Front-end

Streamlit is utilized here for a dashboard monitoring the data being loaded

pip install psutil streamlit 

import streamlit as st

Challenges we ran into

Managing team catchups across three time zones poses a significant challenge for us.

Accomplishments that we're proud of

This is a zero-cost and python-only (requires no frontend language like JS) cloud-based data solution designed for real-time data processing. It caters to a range of users, from individual quantitative traders/programmers to startups seeking for innovative data solutions.

What we learned

Processing data directly via WebSocket and ETL

What's next for PRC

Will enhance the automation of the database setup, specifically the table creation process, by integrating LLM APIs. Additionally, will modify the Streamlit app to facilitate input for new websockets.

Built With

• amazon-web-services
• python
• sqlalchemy
• websockets