DEVELOPER_GUIDE.md

MySQL Socket Event Server - Developer Guide

📋 Table of Contents

Project Overview
Architecture and Design
Installation and Configuration
Project Structure
Core Components
API Reference
Security
Development Process
Testing and Debug
Deployment
Troubleshooting

🎯 Project Overview

What It Does

MySQL Socket Event Server is a Node.js application that delivers database changes (insert, update, delete) to clients in real-time via WebSocket connections.

Core Features

Real-time Event Broadcasting: Database changes are delivered instantly
JWT Authentication: Token-based secure authentication
Hierarchical Channel System: Flexible channel structure for targeted listening
Table-based Authorization: Users can only listen to tables they are authorized for
Rate Limiting: DDoS protection and resource management
Modern Web UI: Ready-to-use interface for testing and development

Technology Stack

Backend: Node.js, Express.js
WebSocket: Socket.io
Authentication: JSON Web Tokens (JWT)
Frontend: Vanilla JavaScript, Bootstrap 5

🏗️ Architecture and Design

System Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Web Client    │    │   Mobile App    │    │   Backend API   │
│                 │    │                 │    │                 │
└─────────┬───────┘    └─────────┬───────┘    └─────────┬───────┘
          │                      │                      │
          └──────────────────────┼──────────────────────┘
                                 │
                    ┌─────────────┴─────────────┐
                    │   MySQL Socket Server    │
                    │                           │
                    │  ┌─────────────────────┐  │
                    │  │   Socket.io Layer   │  │
                    │  └─────────────────────┘  │
                    │  ┌─────────────────────┐  │
                    │  │   Express.js API    │  │
                    │  └─────────────────────┘  │
                    │  ┌─────────────────────┐  │
                    │  │   JWT Auth Layer    │  │
                    │  └─────────────────────┘  │
                    └───────────────────────────┘

Channel Hierarchy

db (root)
├── db.users
│   ├── db.users.insert
│   ├── db.users.update
│   │   └── db.users.update.123
│   └── db.users.delete
│       └── db.users.delete.456
├── db.products
│   └── db.products.*
│       └── db.products.*.789
└── db.*.insert (wildcard)

Data Flow

Event Generation: Application detects database change
Event Publishing: dbChange event is sent to Socket.io
Channel Resolution: Event is distributed to hierarchical channels
Authorization Check: User permissions are checked
Event Delivery: Event is delivered to authorized clients

⚙️ Installation and Configuration

System Requirements

Node.js v16+ (recommended v18+)
npm v8+ or yarn v1.22+
512MB+ RAM
100MB+ disk space

Step-by-Step Installation

1. Download Project

git clone <repository-url>
cd universaldb-socket

2. Install Dependencies

npm install
# or
yarn install

3. Configure Environment Variables

cp .env.example .env

Edit .env file:

# Server Configuration
PORT=3001
NODE_ENV=development

# JWT Configuration (IMPORTANT: Change in production!)
JWT_SECRET=your-super-secret-jwt-key-here
JWT_EXPIRES_IN=24h

# Rate Limiting
RATE_LIMIT_WINDOW_MS=60000
RATE_LIMIT_MAX_REQUESTS=100

# CORS Configuration
CORS_ORIGIN=http://localhost:3001

4. Start Server

# Development mode (auto-reload)
npm run dev

# Production mode
npm start

5. Test

Open http://localhost:3001 in browser
Use http://localhost:3001/index.html file for testing
Visit http://localhost:3001/monitor.html for monitoring

📁 Project Structure

universaldb-socket/
├── src/                          # Main source code
│   ├── app.js                    # Main application class
│   ├── config/                   # Configuration files
│   │   ├── server.js             # Server settings
│   │   └── jwt.js                # JWT configuration
│   ├── controllers/              # Business logic controllers
│   │   ├── socketController.js   # Socket.io event handlers
│   │   └── apiController.js      # REST API handlers
│   ├── middleware/               # Middleware
│   │   ├── auth.js               # Authentication
│   │   ├── rateLimiter.js        # Rate limiting
│   │   └── errorHandler.js       # Error handling
│   ├── services/                 # Business logic services
│   │   ├── eventService.js       # Event publishing
│   │   └── socketService.js      # Socket management
│   ├── utils/                    # Utility tools
│   │   ├── logger.js             # Logging
│   │   └── validator.js          # Data validation
│   └── routes/                   # API routes
│       └── api.js                # REST endpoints
├── public/                       # Static files
│   ├── index.html                # Main page
│   ├── monitor.html              # Monitoring page
│   ├── css/
│   └── js/
├── server.js                     # Application entry point
├── package.json                  # Project metadata
├── .env.example                  # Environment variables template

└── README.md                     # Project documentation

🔧 Core Components

1. Application Class (`src/app.js`)

Main application class. Configures Express.js and Socket.io.

class Application {
    constructor() {
        this.app = express();
        this.server = http.createServer(this.app);
        this.io = new Server(this.server, config.socketio);
    }
    
    start() {
        this.server.listen(config.port);
    }
}

Responsibilities:

Express.js middleware setup
Socket.io configuration
Route definitions
Error handling
Graceful shutdown

2. Socket Controller (`src/controllers/socketController.js`)

Manages Socket.io events.

class SocketController {
    static handleDbChange(io, socket, data) {
        // Rate limiting
        // Event validation
        // Event publishing
        // Response sending
    }
}

Responsibilities:

Processing dbChange events
Rate limiting control
Event validation
Response sending

3. Event Service (`src/services/eventService.js`)

Manages event publishing logic.

class EventService {
    static publishDbChange(io, data) {
        const events = this.generateEventChannels(table, action, record);
        events.forEach(({ channel, eventName }) => {
            io.to(channel).emit(eventName, data);
        });
    }
}

Responsibilities:

Hierarchical channel creation
Event distribution
Validation

4. Socket Service (`src/services/socketService.js`)

Socket connection management and channel subscriptions.

class SocketService {
    static handleSubscribe(socket, data) {
        const authorizedChannels = this.getAuthorizedChannels(socket, channel);
        authorizedChannels.forEach(ch => socket.join(ch));
    }
}

Responsibilities:

Socket connection management
Channel subscription operations
Authorization control
Wildcard pattern resolution

5. Auth Middleware (`src/middleware/auth.js`)

JWT-based authentication.

class AuthMiddleware {
    static socketAuth(socket, next) {
        const token = socket.handshake.auth?.token;
        const decoded = jwt.verify(token, jwtConfig.secret);
        socket.user = decoded;
        socket.tables = decoded.tables ? decoded.tables.split(',') : [];
    }
}

Responsibilities:

JWT token validation
Adding user information to socket
Extracting table permissions

📡 API Reference

Socket.io Events

Client → Server

`dbChange`

Database change notification.

socket.emit('dbChange', {
    timestamp: '2024-01-15T10:30:45.123Z',
    table: 'users',
    action: 'update',
    record: {
        id: 123,
        name: 'John Doe',
        email: '[email protected]'
    }
});

Parameters:

timestamp (string): ISO 8601 format
table (string): Table name
action (string): 'insert', 'update', 'delete'
record (object): Record data

`subscribe`

Subscribe to channel.

socket.emit('subscribe', 'db.users.update', (authorizedChannels) => {
    console.log('Subscribed channels:', authorizedChannels);
});

Parameters:

channel (string): Channel name
callback (function): Success callback

`unsubscribe`

Unsubscribe from channel.

socket.emit('unsubscribe', 'db.users.update');

Server → Client

`db.*` (Hierarchical Events)

Database change notifications.

socket.on('db.users.update', (data) => {
    console.log('User updated:', data);
});

`subscribed`

Subscription success notification.

socket.on('subscribed', (data) => {
    console.log('Subscribed:', data.authorizedChannels);
});

`error`

Error notification.

socket.on('error', (error) => {
    console.error('Socket error:', error.message);
});

Channel Formats

Format	Description	Example
`db`	All changes	All events
`db.[table]`	Table-based	`db.users`
`db.[table].[action]`	Table + action	`db.users.insert`
`db.[table].[action].[id]`	Specific record	`db.users.update.123`
`db.[table].*.[id]`	Record-based (all actions)	`db.users.*.123`
`db.*.[action]`	Action-based (all tables)	`db.*.delete`

🔐 Güvenlik

JWT Token Structure

1. User JWT Token (Socket Connections)

{
  "sub": "user_id",           // User ID
  "name": "User Name",        // User name
  "tables": "table1,table2",  // Accessible tables (comma-separated)
  "iat": 1640995200,           // Token creation time
  "exp": 1640998800            // Token expiration time
}

Field Descriptions:

sub: User's unique identifier
name: User's display name
tables: List of tables the user can access
iat: Token creation time (Unix timestamp)
exp: Token expiration time (Unix timestamp)

2. Admin or Publisher JWT Token (External Systems)

For DB change requests from external systems:

{
  "sub": "user_id",           // Admin system ID
  "name": "User Name",        // Admin system name
  "admin": true,              // Admin privilege
  "publisher": false,         // Publisher privilege
  "iat": 1753215601,          // Token creation time
  "exp": 1753219201           // Token expiration time
}

Usage Scenarios:

DB change notifications from CodeIgniter applications
Real-time updates from Laravel API
Event sending from Node.js microservices
Data synchronization from other backend systems

Security Measures

1. JWT Secret Security

# Must change in production!
export JWT_SECRET="$(openssl rand -base64 32)"

2. CORS Configuration

// config/server.js
socketio: {
    cors: {
        origin: process.env.CORS_ORIGIN || 'http://localhost:3001',
        methods: ['GET', 'POST'],
        credentials: true
    }
}

3. Rate Limiting

// middleware/rateLimiter.js
const limits = new Map();

class RateLimiter {
    static check(userId, maxRequests) {
        // Implementation
    }
}

4. Input Validation

// utils/validator.js
static validateEventData(data) {
    // Timestamp validation
    // Table name validation
    // Action validation
    // Record validation
}

🛠️ Development Process

Development Environment Setup

1. Required Tools

# Node.js version manager
nvm install 18
nvm use 18

# Global tools
npm install -g nodemon eslint prettier

2. IDE Configuration

VS Code Extensions:

ES6 String HTML
Prettier
ESLint
Node.js Extension Pack

3. Git Hooks

# Pre-commit hook
npm run lint
npm run format

Code Standards

1. ESLint Configuration

{
  "extends": ["eslint:recommended"],
  "env": {
    "node": true,
    "es2021": true
  },
  "rules": {
    "indent": ["error", 4],
    "quotes": ["error", "single"],
    "semi": ["error", "always"]
  }
}

2. Prettier Configuration

{
  "tabWidth": 4,
  "singleQuote": true,
  "trailingComma": "es5",
  "printWidth": 100
}

3. Naming Conventions

Classes: PascalCase (EventService)
Functions: camelCase (handleDbChange)
Constants: UPPER_SNAKE_CASE (JWT_SECRET)
Files: camelCase (socketController.js)

Adding New Features

1. Adding Event Handler

// src/controllers/socketController.js
static handleNewEvent(io, socket, data) {
    try {
        // Validation
        // Business logic
        // Response
    } catch (error) {
        // Error handling
    }
}

2. Adding Middleware

// src/middleware/newMiddleware.js
class NewMiddleware {
    static process(req, res, next) {
        // Implementation
        next();
    }
}

3. Adding Service

// src/services/newService.js
class NewService {
    static processData(data) {
        // Implementation
    }
}

🧪 Testing and Debug

Testing Strategy

1. Unit Tests

// tests/services/eventService.test.js
const EventService = require('../../src/services/eventService');

describe('EventService', () => {
    test('should generate correct channels', () => {
        const channels = EventService.generateEventChannels('users', 'update', {id: 123});
        expect(channels).toContain('db.users.update.123');
    });
});

2. Integration Tests

// tests/integration/socket.test.js
const io = require('socket.io-client');

describe('Socket Integration', () => {
    test('should authenticate with valid token', (done) => {
        const socket = io('http://localhost:3001', {
            auth: { token: validToken }
        });
        socket.on('connect', done);
    });
});

Debug Techniques

1. Logging

// src/utils/logger.js
const logger = require('./logger');

logger.info('Connection established', { socketId: socket.id });
logger.error('Authentication failed', { error: error.message });
logger.debug('Event data', { data });

2. Socket.io Debug

# All Socket.io debug information
DEBUG=socket.io:* node server.js

# Server debug only
DEBUG=socket.io:server node server.js

3. Chrome DevTools

// In browser console
socket.on('connect', () => console.log('Connected'));
socket.on('disconnect', () => console.log('Disconnected'));
socket.onAny((event, ...args) => console.log(event, args));

Testing Tools

1. Using Emit.html

Open http://localhost:3001/index.html
Update JWT token
Perform event sending and listening tests

2. Postman/Insomnia

// WebSocket connection test
const socket = new WebSocket('ws://localhost:3001/socket.io/?EIO=4&transport=websocket');

3. Node.js Test Script

// test-client.js
const io = require('socket.io-client');

const socket = io('http://localhost:3001', {
    auth: { token: 'your-jwt-token' }
});

socket.on('connect', () => {
    console.log('Connected');
    socket.emit('subscribe', 'db.users');
});

🚀 Deployment

Production Preparation

1. Environment Variables

# .env.production
NODE_ENV=production
PORT=3001
JWT_SECRET=your-super-secure-production-secret
CORS_ORIGIN=https://yourdomain.com
LOG_LEVEL=warn

2. PM2 Configuration

// ecosystem.config.js
module.exports = {
    apps: [{
        name: 'universaldb-socket-server',
        script: 'server.js',
        instances: 'max',
        exec_mode: 'cluster',
        env: {
            NODE_ENV: 'development'
        },
        env_production: {
            NODE_ENV: 'production',
            PORT: 3001
        }
    }]
};

3. Docker Deployment

# Dockerfile
FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY . .

EXPOSE 3001

USER node

CMD ["node", "server.js"]

# docker-compose.yml
version: '3.8'
services:
  universaldb-socket:
    build: .
    ports:
      - "3001:3001"
    environment:
      - NODE_ENV=production
      - JWT_SECRET=${JWT_SECRET}
    restart: unless-stopped

Monitoring and Logging

1. Health Check Endpoint

// src/routes/api.js
router.get('/health', (req, res) => {
    res.json({
        status: 'ok',
        timestamp: new Date().toISOString(),
        uptime: process.uptime(),
        memory: process.memoryUsage()
    });
});

2. Metrics Collection

// src/utils/metrics.js
class Metrics {
    static trackConnection(socketId) {
        // Implementation
    }
    
    static trackEvent(eventType) {
        // Implementation
    }
}

🔍 Troubleshooting

Common Issues and Solutions

1. Connection Issues

Issue: Cannot establish socket connection

Error: xhr poll error

Solution:

// Check CORS settings
// config/server.js
socketio: {
    cors: {
        origin: "*", // For development
        methods: ["GET", "POST"]
    }
}

2. Authentication Issues

Issue: JWT token invalid

Error: Invalid authentication token

Solution:

// Check the token
const jwt = require('jsonwebtoken');
const decoded = jwt.verify(token, process.env.JWT_SECRET);
console.log(decoded);

3. Event Delivery Issues

Issue: Events are not being delivered

Debug Steps:

// 1. Check socket rooms
console.log(io.sockets.adapter.rooms);

// 2. Check user permissions
console.log(socket.tables);

// 3. Check channel authorization
const authorized = SocketService.isAuthorizedForChannel(socket, channel);
console.log('Authorized:', authorized);

4. Performance Issues

Issue: Slow response times

Optimization:

// 1. Check rate limiting settings
// 2. Reduce event payload size
// 3. Clean up unnecessary channel subscriptions
// 4. Check for memory leaks

Debug Commands

# Detailed logging
DEBUG=* npm run dev

# Socket.io specific
DEBUG=socket.io:* npm run dev

# Memory usage monitoring
node --inspect server.js

# Performance profiling
node --prof server.js

Log Analysis

# Filter error logs
grep "ERROR" logs/app.log

# Count connection logs
grep "Socket connected" logs/app.log | wc -l

# Find rate limit violations
grep "Rate limit exceeded" logs/app.log

📚 Additional Resources

Documentation

Development Tools

Postman - API testing
Socket.io Client Tool - WebSocket testing
JWT Debugger - Token debugging

Monitoring Tools

Last Updated: 2024-01-15
Version: 1.0.0
Developer: Şafak Bahçe

This documentation is prepared for developers new to the project. You can use the GitHub Issues page for your questions.

FilesExpand file tree

DEVELOPER_GUIDE.md

Latest commit

History

DEVELOPER_GUIDE.md

File metadata and controls

MySQL Socket Event Server - Developer Guide

📋 Table of Contents

🎯 Project Overview

What It Does

Core Features

Technology Stack

🏗️ Architecture and Design

System Architecture

Channel Hierarchy

Data Flow

⚙️ Installation and Configuration

System Requirements

Step-by-Step Installation

1. Download Project

2. Install Dependencies

3. Configure Environment Variables

4. Start Server

5. Test

📁 Project Structure

🔧 Core Components

1. Application Class (src/app.js)

2. Socket Controller (src/controllers/socketController.js)

3. Event Service (src/services/eventService.js)

4. Socket Service (src/services/socketService.js)

5. Auth Middleware (src/middleware/auth.js)

📡 API Reference

Socket.io Events

Client → Server

dbChange

subscribe

unsubscribe

Server → Client

db.* (Hierarchical Events)

subscribed

error

Channel Formats

🔐 Güvenlik

JWT Token Structure

1. User JWT Token (Socket Connections)

2. Admin or Publisher JWT Token (External Systems)

Security Measures

1. JWT Secret Security

2. CORS Configuration

3. Rate Limiting

4. Input Validation

🛠️ Development Process

Development Environment Setup

1. Required Tools

2. IDE Configuration

3. Git Hooks

Code Standards

1. ESLint Configuration

2. Prettier Configuration

3. Naming Conventions

Adding New Features

1. Adding Event Handler

2. Adding Middleware

3. Adding Service

🧪 Testing and Debug

Testing Strategy

1. Unit Tests

2. Integration Tests

Debug Techniques

1. Logging

2. Socket.io Debug

3. Chrome DevTools

Testing Tools

1. Using Emit.html

2. Postman/Insomnia

3. Node.js Test Script

🚀 Deployment

Production Preparation

1. Environment Variables

2. PM2 Configuration

1. Application Class (`src/app.js`)

2. Socket Controller (`src/controllers/socketController.js`)

3. Event Service (`src/services/eventService.js`)

4. Socket Service (`src/services/socketService.js`)

5. Auth Middleware (`src/middleware/auth.js`)

`dbChange`

`subscribe`

`unsubscribe`

`db.*` (Hierarchical Events)

`subscribed`

`error`