Overview

Relevant source files

• Cargo.lock
• Cargo.toml
• README.md
• docs/en/ContributionGuide/ContributingCode/Code-Structure.md
• docs/images/wechat-group.jpg
• docs/images/wechat.jpg
• docs/zh/ContributionGuide/ContributingCode/Code-Structure.md
• makefile
• src/admin-server/src/mqtt/connector.rs
• src/broker-server/src/lib.rs
• src/common/base/Cargo.toml
• src/common/base/src/error/common.rs
• src/common/metadata-struct/src/mqtt/topic_rewrite_rule.rs
• src/common/network-server/src/common/metric.rs
• src/mqtt-broker/Cargo.toml
• src/mqtt-broker/src/broker.rs

Purpose and Scope

This page provides a high-level introduction to RobustMQ, a next-generation unified messaging infrastructure built in Rust. It covers:

• System vision and design goals
• Core architectural components and their relationships
• Key features and capabilities
• Current development status

For detailed information about specific subsystems, see:

• System Architecture → 2
• MQTT Broker implementation → 3
• Storage Layer → 4
• Meta Service and cluster coordination → 5
• Connector system for data integration → 6

Sources: README.md1-175 docs/zh/ContributionGuide/ContributingCode/Code-Structure.md1-280

---

System Vision

RobustMQ is designed as a unified messaging infrastructure that consolidates multiple messaging protocols and storage strategies into a single platform. The primary design goals are:

1. Multi-Protocol Unification: Support MQTT (3.x/5.0) and Kafka protocols on the same data plane, eliminating the need to maintain separate messaging systems
2. AI-Ready Data Path: Multi-tier caching (memory/SSD/object storage) and object storage integration (S3/MinIO) to optimize data loading for AI training workloads
3. Edge-to-Cloud Consistency: Single architecture deployable from edge gateways to cloud clusters with offline buffering and sync capabilities
4. Operational Simplicity: Single-binary deployment with built-in Raft consensus and pluggable storage backends

Target Workloads

Workload Type	Use Case	Key Feature
AI	Agent communication, training data ingestion	Lightweight topics, shared subscriptions, storage tiering
IoT	Device telemetry, command/control	MQTT ingestion with Kafka consumption on same data plane
Data	Stream processing, analytics	Flexible storage modes (memory/hybrid/persistent/tiered)

Sources: README.md41-59 README.md61-73

---

System Architecture

RobustMQ implements a three-tier architecture with isolated runtime execution contexts:

Key Architectural Decisions

1. Three Isolated Runtimes: The BrokerServer creates three separate Tokio runtimes to prevent resource contention between gRPC handling, Raft consensus operations (~9 internal tasks per shard), and protocol message processing
2. Raft-Based Coordination: The MetaServiceServer uses multi-Raft groups (metadata, offset, data) for distributed consensus
3. Pluggable Storage: The StorageDriverManager and StorageAdapter trait enable memory, RocksDB, MySQL, PostgreSQL, and S3 backends
4. Connection Pooling: ClientPool manages gRPC client connections with configurable channels per address

Sources: src/broker-server/src/lib.rs86-108 src/broker-server/src/lib.rs116-238 Architecture diagrams

---

Core Components

Component Hierarchy

Component Responsibilities

Component	File Path	Primary Responsibility
BrokerServer	src/broker-server/src/lib.rs86-239	Main server orchestration, runtime management, component lifecycle
MetaServiceServer	src/meta-service/	Cluster metadata storage, Raft consensus, node registration/heartbeat
MqttBrokerServer	src/mqtt-broker/src/broker.rs70-345	MQTT protocol handling, session management, QoS delivery
StorageEngineServer	src/storage-engine/	Message persistence, segment-based file storage, offset tracking
AdminServer	src/admin-server/	HTTP REST API for cluster management and monitoring
ClientPool	src/grpc-clients/	gRPC client connection pooling and load balancing
StorageDriverManager	src/storage-adapter/	Pluggable storage abstraction, per-topic storage routing
ConnectorManager	src/connector/	Data egress to 17+ external systems (Kafka, databases, etc.)

Sources: src/broker-server/src/lib.rs86-239 src/mqtt-broker/src/broker.rs50-128 Cargo.toml304-350

---

Key Features

Protocol Support

RobustMQ implements multiple messaging protocols with protocol-specific optimizations:

Protocol	Status	Implementation	Key Features
MQTT 3.x/5.0	✅ Stable	src/mqtt-broker/	QoS 0/1/2, retained messages, shared subscriptions, session persistence
Kafka	🚧 In Progress	src/kafka-broker/	Protocol compatibility for AI training data ingestion
AMQP	🚧 Planned	src/amqp-broker/	Enterprise messaging patterns

Network transport options:

• TCP: Standard protocol transport via src/mqtt-broker/src/server/tcp/
• TLS: Encrypted connections via tokio-rustls
• WebSocket: Browser-compatible transport
• QUIC: UDP-based low-latency transport via quinn

Sources: README.md76-77 src/mqtt-broker/Cargo.toml1-93 Cargo.toml148-153

Storage Architecture

Storage Configuration Options:

• Memory: Low latency, no persistence (development/testing)
• RocksDB: Local SSD persistence with multi-threaded compaction
• Hybrid: Hot data in memory, cold data on disk
• Tiered: Object storage (S3/MinIO) for archival with local cache

The StorageDriverManager routes storage operations per-topic based on configuration, enabling different topics to use different storage strategies within the same cluster.

Sources: src/storage-adapter/ src/storage-engine/ Architecture Diagram 4

Connector System

RobustMQ provides data egress connectors to integrate with external systems:

Category	Connectors	Configuration
Message Queues	Kafka, Pulsar, RabbitMQ, MQTT Bridge	src/connector/config_kafka.rs src/connector/config_pulsar.rs
Databases	MySQL, PostgreSQL, MongoDB, Cassandra	src/connector/config_mysql.rs src/connector/config_mongodb.rs
Time Series	GreptimeDB, InfluxDB, OpenTSDB	src/connector/config_greptimedb.rs src/connector/config_influxdb.rs
Search/Analytics	Elasticsearch, ClickHouse	src/connector/config_elasticsearch.rs src/connector/config_clickhouse.rs
Storage	S3/MinIO, Local File, Redis	src/connector/config_s3.rs src/connector/config_local_file.rs
Other	Webhook	src/connector/config_webhook.rs

Each connector runs in a dedicated thread managed by ConnectorManager with metrics tracking (send_success_total, send_fail_total). Failure handling strategies include:

• Discard: Drop failed messages immediately
• DiscardAfterRetry: Retry with configurable count and backoff
• DeadMessageQueue: Route to dead letter topic after retry exhaustion

Sources: src/admin-server/src/mqtt/connector.rs1-448 Architecture Diagram 7

Security and Authentication

The AuthManager provides pluggable authentication with multiple backend support:

Authentication Mode	Backend	Configuration
Secret-Free	No authentication	Development mode
Password-Based	MySQL, Placement Storage, Redis	Configurable via auth_driver
Authorization	ACL validation, blacklist checking	Topic-level permissions

Hash algorithms supported: MD5, SHA1, SHA256, bcrypt, PBKDF2, HMAC-SHA256

Sources: src/mqtt-broker/src/security/ src/mqtt-broker/Cargo.toml74-86 Architecture Diagram 6

---

Multi-Runtime Architecture

RobustMQ uses three isolated Tokio runtimes to prevent task scheduler contention:

Runtime Configuration

Worker thread counts are configurable and independently tuned:

Runtime	Config Key	Purpose	Default Calculation
server_runtime	runtime.server_worker_threads	gRPC, HTTP, admin tasks	num_cpus() / 4
meta_runtime	runtime.meta_worker_threads	Raft consensus operations	num_cpus() / 4
broker_runtime	runtime.broker_worker_threads	Protocol message processing	num_cpus() / 2

The blog documents that 16 threads per runtime achieved 20k+ ops/s on a 14-CPU machine, demonstrating that slight oversubscription improves throughput.

Critical Design Decision: Raft's Raft::new() is called inside meta_runtime.block_on() so all openraft internal tasks (approximately 9 per shard: core loop, log I/O, state machine, snapshot builder, replication, heartbeat) are spawned on the meta runtime. This prevents interference with latency-sensitive protocol handling on the broker runtime.

Sources: src/broker-server/src/lib.rs126-131 src/broker-server/src/lib.rs161-193 common/base/src/runtime.rs Architecture Diagram 2

---

Development Status

RobustMQ follows a phased development roadmap:

Phase	Status	Focus	Deliverables
Phase 1	✅ Completed	Foundation	Scalable architecture, pluggable storage, multi-protocol support
Phase 2	✅ Initial Release	MQTT Broker	MQTT 3.x/5.0 support, <20MB binary for edge deployment
Phase 3	🚧 In Progress	Kafka Protocol & AI	AI training data caching, million-scale lightweight topics, Kafka compatibility

Current Capabilities

• Production-Ready: MQTT 3.x/5.0 broker with QoS 0/1/2 support
• Storage Options: Memory, RocksDB, MySQL, PostgreSQL, S3/MinIO
• Cluster Management: Raft-based metadata service with automatic failover
• Data Integration: 17+ connectors for external system integration
• Observability: Prometheus metrics, system topic monitoring, pprof profiling

Under Development

• Kafka Protocol: Standard Kafka API compatibility for AI workloads
• Lightweight Topics: Million-scale topic support with per-topic isolation
• AI Data Acceleration: Multi-tier caching for training data loading

Sources: README.md61-73 README.md38-39

---

Build and Deployment

Single-Binary Deployment

RobustMQ compiles to a single binary (broker-server) that includes all components:

Configuration

Main configuration file: config/server.toml

Key configuration sections:

• cluster: Cluster name, node ID, roles (broker/meta/engine)
• grpc_port, http_port: Service ports
• runtime: Worker thread counts for each runtime
• rocksdb: RocksDB storage settings
• prometheus: Metrics collection settings
• p_prof: Performance profiling settings

Sources: README.md85-117 makefile28-42 src/broker-server/src/lib.rs240-435

---

Related Documentation

For detailed information about specific subsystems:

• Architecture Details: System Architecture

  • Four-Layer Architecture - Application, protocol, service, and foundation layers
  • Multi-Runtime Architecture - Isolated Tokio runtimes and performance
  • Multi-Protocol Support - TCP, TLS, WebSocket, QUIC
  • Service Communication - gRPC interfaces and client pooling

• MQTT Broker: MQTT Broker

  • Server Initialization - Component lifecycle and bootstrapping
  • Network Protocol Handlers - TCP, TLS, WS, QUIC servers
  • MQTT Packet Processing - Encoding/decoding and command handling
  • Connection Management - Connection lifecycle and state tracking
  • Session Management - Session persistence and clean session behavior
  • Subscription Management - Topic-to-subscriber mapping
  • Message Delivery - PushManager and QoS handling
  • Security - Authentication and authorization

• Storage Layer: Storage Layer

  • Storage Architecture - Multi-tier storage design
  • StorageAdapter Interface - Pluggable backend pattern
  • Storage Engine Server - Segment-based file storage
  • Message Storage - Persistence and sharding
  • Offset Management - Consumer group tracking

• Meta Service: Meta Service

  • Multi-Raft Design - Three Raft groups architecture
  • Raft Consensus - Leader election and log replication
  • Snapshot Management - Snapshot creation and recovery
  • Metadata Operations - gRPC API for metadata CRUD
  • Broker Registration - Node heartbeat mechanism

• Connector System: Bridge and Connector System

  • Connector Architecture - Lifecycle management and metrics
  • Message Processing Pipeline - ETL transformation and batch sending
  • Connector Types - 17+ external system integrations
  • Failure Handling - Retry strategies and dead letter queue

• Operations: Management and Operations

  • Admin HTTP API - REST endpoints for management
  • CLI Tools - robust-ctl command reference
  • Monitoring and Metrics - Prometheus integration

• Build and Test: Build, Test, and Deployment

  • Build System - Makefile targets and compilation
  • Testing Infrastructure - Unit and integration tests
  • CI/CD Workflows - GitHub Actions automation
  • Configuration Management - server.toml reference
  • Deployment Options - Performance tuning strategies

Sources: Table of contents JSON structure