Skip to content

EZFRICA/cloudsql-postgres-manager

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

33 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Cloud SQL PostgreSQL Manager

A comprehensive solution for automating Google Cloud SQL PostgreSQL database management and IAM user permissions across multiple databases in an organization.

🎬 Application Demo

This application provides a complete REST API for managing PostgreSQL databases, schemas, roles, and IAM users. Below you'll find screenshots demonstrating the key features:

Quick Overview

  • Health Monitoring: Real-time service and database health checks
  • Schema Management: Create and manage database schemas with proper ownership
  • Role Management: Initialize, assign, and revoke PostgreSQL roles
  • IAM Integration: Seamless Google Cloud IAM user management
  • Database Operations: List schemas, tables, and perform health checks

Scroll down to see detailed screenshots for each feature!

πŸ“š Documentation Overview

This project contains comprehensive documentation for the Cloud SQL PostgreSQL Manager, a modular FastAPI service for managing PostgreSQL databases, schemas, roles, and IAM user permissions in Google Cloud SQL.

πŸ“– Documentation Structure

πŸ—οΈ Architecture Documentation

🌐 API Documentation

πŸ§ͺ Testing Documentation

πŸš€ Deployment Documentation

🎯 Quick Start

1. Architecture Understanding

Start with md/ARCHITECTURE_OVERVIEW.md to understand the system design and components.

2. Service Details

Read md/SERVICES.md to understand each service's responsibilities and interactions.

3. API Usage

Check md/API.md for endpoint documentation and examples.

4. Deployment

Follow md/DEPLOYMENT.md for deployment instructions.

πŸ—οΈ System Architecture

The system follows a modular microservices architecture with clear separation of concerns:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   API Layer     β”‚    β”‚  Service Layer  β”‚    β”‚ Component Layer β”‚
β”‚                 β”‚    β”‚                 β”‚    β”‚                 β”‚
β”‚ β€’ Health Router β”‚    β”‚ β€’ ConnectionMgr β”‚    β”‚ β€’ Validation    β”‚
β”‚ β€’ Database Routerβ”‚    β”‚ β€’ SchemaMgr    β”‚    β”‚ β€’ ErrorHandler  β”‚
β”‚ β€’ Schema Router β”‚    β”‚ β€’ RoleMgr       β”‚    β”‚ β€’ ServiceOps    β”‚
β”‚ β€’ Role Router   β”‚    β”‚ β€’ UserMgr       β”‚    β”‚ β€’ DatabaseOps   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ”§ Key Features

Database Management

  • Schema Operations: Create, list, and manage database schemas
  • Table Management: List tables with metadata and statistics
  • Health Monitoring: Comprehensive database health checks
  • Connection Pooling: High-performance connection management

Role-Based Access Control

  • Plugin System: Extensible role definitions with versioning
  • Permission Levels: readonly, readwrite, admin with granular control
  • IAM Integration: Seamless Google Cloud IAM user management
  • Role Registry: Firestore-based role tracking and history

Security & Validation

  • Input Validation: Comprehensive request validation
  • IAM Validation: Service account and permission verification
  • SQL Injection Protection: Parameterized queries and sanitization
  • Error Handling: Secure error responses without information leakage

πŸ“‹ Service Overview

Service Purpose Key Features
ConnectionManager Database connection pooling High-performance, automatic recovery
SchemaManager Schema and table operations Creation, listing, ownership management
RoleManager Role initialization and management Plugin system, versioning, Firestore integration
UserManager IAM user operations Validation, normalization, permission checks
RolePermissionManager Role assignments User-role mapping, permission management
HealthManager System monitoring Health checks, performance metrics

πŸ”Œ Plugin System

The system includes an extensible plugin architecture for role management:

  • StandardRolePlugin: Built-in role definitions
  • CustomRolePlugin: Custom role implementations
  • PluginRegistry: Plugin management and loading
  • Version Control: Role versioning and updates

🌐 API Endpoints & Screenshots

Health Monitoring

  • GET /health - Service health check

Health Check

Database Management

  • POST /database/schemas - List database schemas
  • POST /database/tables - List schema tables
  • POST /database/health - Database health check

List all schemas in the database

List all tables in a specific schema

Check database health and connection status

Schema Management

  • POST /schemas/create - Create database schema

Create schemas

Role Management

  • POST /roles/initialize - Initialize roles
  • POST /roles/assign - Assign role to user
  • POST /roles/revoke - Revoke role from user
  • POST /roles/list - List available roles

Initialize PostgreSQL roles

Assign a role to a user

Revoke a role from a user

Get all users and their assigned roles

IAM User Management

  • POST /database/postgres-inheritance/grant - Grant IAM user to postgres
  • POST /database/postgres-inheritance/revoke - Revoke IAM user from postgres

Grant an IAM user to postgres

Revoke an IAM user from postgres

πŸ“Š PostgreSQL Results

Database Schema Overview

Table in schema

PostgreSQL Command Line Results

psql du - List users

psql dn - List databases

πŸš€ Deployment Options

Local Development

cd postgres-manager
uvicorn app.main:app --reload --host 0.0.0.0 --port 8080

Docker

cd postgres-manager
docker build -t cloudsql-postgres-manager .
docker run -p 8080:8080 cloudsql-postgres-manager

Google Cloud Run

gcloud run deploy cloudsql-postgres-manager \
  --image gcr.io/PROJECT_ID/cloudsql-postgres-manager \
  --platform managed \
  --region europe-west1

πŸ§ͺ Testing

For comprehensive testing documentation, see:

Role Testing

For comprehensive role testing, see the testing documentation:

πŸ“Š Monitoring

Health Checks

  • Service Health: GET /health
  • Database Health: POST /database/health

Metrics

  • Request processing time
  • Database connection metrics
  • Error rates by endpoint
  • Role operation success rates

Logging

Structured JSON logging with correlation IDs and performance metrics.

πŸ”’ Security

Authentication

  • Google Cloud IAM integration
  • Service account validation
  • Permission verification

Data Protection

  • Secret Manager for credentials
  • Parameterized queries
  • Input validation and sanitization

Error Security

  • Sanitized error messages
  • No sensitive data in logs
  • Structured error responses

πŸ“ˆ Performance

Connection Pooling

  • High-performance connection management
  • Configurable pool sizes
  • Automatic connection recovery

Scalability

  • Modular design for independent scaling
  • Plugin-based extensibility
  • Component reusability

πŸ“ Project Structure

cloudsql-postgres-manager/
β”œβ”€β”€ postgres-manager/           # FastAPI application
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ main.py            # Main application code
β”‚   β”‚   β”œβ”€β”€ models.py          # Pydantic models
β”‚   β”‚   β”œβ”€β”€ services/          # Business logic
β”‚   β”‚   β”œβ”€β”€ components/        # Reusable components
β”‚   β”‚   β”œβ”€β”€ routers/           # API endpoints
β”‚   β”‚   β”œβ”€β”€ plugins/           # Plugin system
β”‚   β”‚   └── utils/             # Utilities
β”‚   β”œβ”€β”€ requirements.txt       # Python dependencies
β”‚   └── Dockerfile            # Container configuration
β”œβ”€β”€ md/                        # Documentation
β”‚   β”œβ”€β”€ README.md             # This file
β”‚   β”œβ”€β”€ API.md                # API documentation
β”‚   β”œβ”€β”€ SERVICES.md           # Service documentation
β”‚   β”œβ”€β”€ ROLE_TESTING.md       # Role testing guide
β”‚   └── ...                   # Other documentation
└── README.md                 # This file

🀝 Contributing

  1. Read the architecture documentation
  2. Understand the service responsibilities
  3. Follow the component patterns
  4. Add comprehensive tests
  5. Update documentation

πŸ“ž Support


πŸ”„ Documentation Updates

This documentation is maintained alongside the codebase. When making changes:

  1. Update relevant documentation files
  2. Ensure examples are current
  3. Test all code examples
  4. Update version numbers
  5. Review for accuracy and completeness

For questions or suggestions about the documentation, please open an issue or submit a pull request.

About

Solution for managing Google CloudSQL PostgreSQL databases and IAM user permissions in db with FastAPI service and Terraform infrastructure.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors