Okay, here is the complete and updated README.md that reflects the project's current state, including all the modifications and improvements we've discussed. You can copy and paste this directly into your project's README.md file.
# MSLMR_TRDNG: Advanced Algorithmic Trading System
## Descripción
**MSLMR_TRDNG** is a sophisticated algorithmic trading system designed for cryptocurrency markets. It supports both traditional technical analysis-based strategies and advanced machine learning models to generate trading signals. The system features robust backtesting capabilities, live trading execution, risk management, performance monitoring with an interactive dashboard, and real-time alerts.
## Key Features
* **Modular Design:** Built with a modular architecture for easy extension and maintenance. Each component (data ingestion, preprocessing, strategies, models, etc.) is designed to be independent and easily replaceable.
* **Multiple Trading Strategies:**
* **Traditional Strategies:**
* RSI-MACD Strategy: Combines Relative Strength Index (RSI) and Moving Average Convergence Divergence (MACD) indicators.
* Moving Average Crossover Strategy: Generates signals based on the crossover of short and long moving averages. Supports simple, exponential, and weighted moving averages.
* **Machine Learning Models:**
* Time Series Forecasting: Predicts future price movements using LSTM, GRU, or Transformer models.
* Market Classification: Classifies the next price movement as up or down using Dense or LSTM networks.
* **Advanced Backtesting:**
* Simulates realistic trading conditions, including commission rates and slippage.
* Calculates comprehensive performance metrics (total trades, win rate, total profit, average profit, max drawdown, Sharpe ratio, Sortino ratio).
* Allows for backtesting multiple strategies across various symbols and timeframes.
* **Live Trading Execution:**
* Connects to cryptocurrency exchanges via the `ccxt` library (supports numerous exchanges).
* Executes trades based on generated signals in real-time.
* Supports dry-run mode for simulated trading without using real capital.
* **Robust Risk Management:**
* Calculates position size dynamically based on account balance, risk per trade, and stop-loss levels.
* Implements dynamic stop-loss using Average True Range (ATR).
* Includes trailing stop-loss functionality to protect profits.
* Validates trades against maximum single trade risk and overall portfolio risk limits.
* Analyzes portfolio concentration and provides alerts if it exceeds a defined threshold.
* Performs Monte Carlo simulations to estimate potential drawdowns.
* **Performance Monitoring and Analytics:**
* Calculates and tracks key performance metrics (total trades, win rate, profit/loss, Sharpe ratio, max drawdown, etc.).
* Generates detailed performance reports in JSON format.
* **Interactive Dashboard:**
* Visualizes trading performance with interactive charts and tables using `Flask` and `Plotly`.
* Displays real-time updates of latest signals, trades, and alerts through AJAX.
* Provides an overview of backtesting results.
* Implements user authentication for security.
* **Real-Time Alerts:**
* Sends alerts via Telegram and email on trade executions and system errors.
* Supports adding more notification channels through an `AlertChannel` interface.
* Stores alerts in the database for later review.
* **Database Integration:**
* Uses PostgreSQL to store market data, indicators, signals, trades, and backtest results.
* Employs SQLAlchemy ORM for database interactions.
* **Caching:**
* Implements a distributed cache using Redis to store market data and calculation results, improving performance.
* Provides a `CacheDecorator` and a `CacheFactory` for flexible cache management.
* **Logging:**
* Comprehensive structured logging using `structlog` and `pythonjsonlogger` to output logs in JSON format for easy analysis.
* Includes `LoggingMixin` and `log_method` decorator for convenient logging in classes and methods.
* **Configuration:**
* Centralized configuration management using a `ConfigManager` class.
* Supports different configuration files for development and production environments.
* Loads sensitive information from environment variables.
* **Asynchronous Operations:**
* Utilizes `asyncio` and `aiohttp` for asynchronous data fetching and trade execution in live trading mode, improving responsiveness.
* **Extensible:**
* Easily add new strategies by inheriting from the `BaseStrategy` class.
* Add new machine learning models by inheriting from the `BaseModel` class.
* Extend with new data sources, exchanges, and alert channels.
* **Testable:**
* Includes unit tests for core components, ensuring code quality and maintainability.
## Project Structure
MSLMR_TRDNG/
├── analytics/ # Performance analysis and reporting
│ ├── __init__.py
│ ├── performance.py # Performance metric calculations
│ └── reporting.py # Report and dashboard generation
├── config/ # Configuration files
│ ├── __init__.py
│ ├── base.py # Base configurations
│ ├── development.py # Development-specific configurations
│ └── production.py # Production-specific configurations
├── core/ # Core components
│ ├── __init__.py
│ ├── cache.py # Caching system (Redis)
│ ├── config.py # Configuration definitions
│ ├── config_manager.py # Configuration manager
│ ├── database.py # Database connection and base model
│ └── database_interactor.py # Database interaction logic
│ └── logging_system.py # Logging setup and utilities
├── data/ # Data handling
│ ├── __init__.py
│ ├── ingestion.py # Data fetching from exchanges (with asynchronous support)
│ ├── models.py # SQLAlchemy models for database tables
│ ├── preprocessing.py# Data cleaning, normalization, and indicator calculation
│ └── storage.py # Data storage management (database, CSV, JSON)
├── models/ # Trading models and strategies
│ ├── __init__.py
│ ├── base.py # Abstract base class for models
│ ├── ml/ # Machine learning models
│ │ ├── __init__.py
│ │ ├── forecasting.py # Time series forecasting models
│ │ ├── model_registry.py # Model registry
│ │ ├── montecarlo.py # Monte Carlo Tree Search (currently a placeholder)
│ │ └── prediction.py # Market classification and regression models
│ └── traditional/ # Traditional technical analysis strategies
│ ├── __init__.py
│ ├── moving_average.py # Moving Average Crossover strategy
│ └── rsi_macd.py # RSI + MACD strategy
├── monitoring/ # Monitoring and alerting
│ ├── __init__.py
│ ├── alerts.py # Alert system (Telegram, Email)
│ └── dashboards.py # Real-time dashboard (Flask)
├── scripts/ # Scripts for training models, running backtests, etc.
│ ├── __init__.py
│ ├── run_backtests.py# Backtesting script
│ └── train_models.py # Model training script
├── strategies/ # Trading strategies and related components
│ ├── __init__.py
│ ├── base.py # Abstract base class for strategies
│ ├── portfolio_manager.py# Portfolio management and position sizing
│ ├── risk_management.py # Risk management functions
│ └── strategy_registry.py# Strategy registry
├── tests/ # Unit tests
│ ├── __init__.py
│ ├── test_models.py # Tests for models
│ ├── test_strategies.py # Tests for strategies
│ └── test_trading.py # Tests for trading execution and backtesting
├── trading/ # Trading execution and backtesting
│ ├── __init__.py
│ ├── backtesting.py # Backtesting engine
│ ├── execution.py # Trade execution logic
│ └── live_trading.py # Live trading system
├── utils/ # Utility functions
│ ├── __init__.py
│ ├── helpers.py # Miscellaneous helper functions
│ └── validators.py # Data validation functions
├── .env.example # Example environment variables file
├── .gitignore # Git ignore rules
├── main.py # Main application entry point
└── requirements.txt # Project dependencies## Installation
1. **Clone the repository:**
```bash
git clone <repository_url>
cd MSLMR_TRDNG
```
2. **Create and activate a virtual environment:**
```bash
python3 -m venv venv
source venv/bin/activate # Linux/Mac
venv\Scripts\activate # Windows
```
3. **Install dependencies:**
```bash
pip install -r requirements.txt
```
## Configuration
1. **Environment Variables:**
* Create a `.env` file in the project's root directory or copy and rename the provided `.env.example`.
* Fill in the necessary values for database connection, exchange API keys, Telegram bot token, email settings, and other configuration parameters as explained in the `.env.example` file.
2. **Configuration Files:**
* The `config/` directory contains configuration files for different environments (base, development, production).
* Modify the settings in these files according to your needs. You can create an additional `testing` configuration if needed.
## Database Setup
The system uses PostgreSQL as its database backend.
1. **Install PostgreSQL:** Follow the instructions for your operating system to install PostgreSQL (version 12 or higher is recommended).
2. **Create Database and User:**
* Create a new database (e.g., `trading_db`).
* Create a new user (e.g., `trading_user`) with a secure password.
* Grant all privileges on the database to the new user.
```sql
CREATE DATABASE trading_db;
CREATE USER trading_user WITH ENCRYPTED PASSWORD 'your_password';
GRANT ALL PRIVILEGES ON DATABASE trading_db TO trading_user;
```
3. **Database Initialization:**
* Run `python init_db.py` to create the necessary tables in the database. This script uses the SQLAlchemy models defined in `data/models.py`.
## Usage
### Running the Live Trading System
To start the live trading system, run:
```bash
python main.py
This will:
1. Initialize the necessary components (configuration, logging, database, strategies, models, etc.).
2. Start the live trading system, which will continuously fetch market data, generate signals, execute trades (in `dry_run` mode if enabled), and monitor performance.
**Note:**
* Ensure that you have configured the `.env` file with your exchange API keys and other settings.
* You can use the `--dry-run` argument to simulate trading without executing real orders.
### Accessing the Dashboard
The dashboard will be available at `http://localhost:8000` (or the configured host and port). You'll need to log in with the configured credentials (currently a placeholder in the code - you need to implement a proper authentication mechanism).
### Backtesting
To run backtests, use the `run_backtests.py` script:
```bash
python scripts/run_backtests.py --start-date 2022-01-01 --end-date 2023-01-01
This will:
1. Run backtests for all registered strategies using the specified date range.
2. Generate performance reports (in JSON format) and an HTML dashboard in the `backtest_results` directory.
You can customize the `start-date`, `end-date`, and other backtesting parameters in the script or through command-line arguments.
### Model Training
To train machine learning models, use the `train_models.py` script:
```bash
python scripts/train_models.py --model-type forecasting --nn-type lstm --epochs 20
This will:
1. Train a TimeSeriesForecaster model of type 'lstm' for 20 epochs.
2. Save the trained model in the `trained_models/` directory.
You can adjust the `model-type`, `nn-type`, `epochs`, and other parameters using command-line arguments.
## Further Development
The `brainstorm_proyect` directory contains further brainstorming ideas and a roadmap for future development, including:
* Integration with advanced models (CNNs, Transformers, GPTs).
* Monte Carlo Tree Search (MCTS) for strategy optimization.
* Reinforcement Learning (RL) for strategy learning.
* Meta-learning for adaptation to changing market conditions.
* High-frequency trading (HFT) capabilities.
* More sophisticated risk management techniques.
* Security enhancements (secrets management, authentication, etc.)
* More tests
## License
This project is licensed under the MIT License.
---
This README provides a comprehensive overview of the **MSLMR_TRDNG** project, its features, installation instructions, usage examples, and future development directions. Remember to replace placeholders like `<repository_url>`, API keys, and other sensitive information with your actual values. Also, implement the suggested improvements and features according to the outlined implementation plan in the previous response. Now you have a good starting point for running and testing your system. I'm ready to move on to **Phase 2** when you are!